Big Data Analytics (Specialisation studies)

The aim of the course is to introduce the student to the different concepts of implementing an analytics process. Students learn the process of problem solving in analytics from data understanding and preprocessing, through modelling choices and implementation until the interpretation, visualization and utilization of the analysis. We will look at typical real-life
applications of analytics. The course will provide hands-on lectures to performing the steps of modeling and analysis.

You learn a practical approach for predicting with Machine Learning with all the steps starting from data acquisition and preparation, search for optimal parameters, to comparison of different methods and evaluation of results. You can employ a linear model for regression and classification, train neural networks, build data features with deep learning, represent and process natural text with numerical methods.

You learn how to lead in turbulent times through data-driven management. You also learn to understand how to become an agent of change for transforming data into insights. People are often visual beings and therefore the focus of the course is on reducing information, through algorithms, that can then be visualized. You develop an understanding of visual analytical methods as a communication medium for business intelligence.

You learn to efficiently handle massive datasets and extract hidden knowledge from data. You understand how to employ classification and clustering algorithms on three different types of data: text, streaming and graph. You acquire knowledge of methods and programming tools for processing big data on distributed/cloud systems.

You are given an overview of machine learning and how to utilize big data. The methods for descriptive and predictive modelling are introduced for small data, and you are then given an explanation for how similar models can be modified to work with big data. You will be introduced to the analytical process; big data tooling, data-related requirement handling, domain knowledge, modelling and verification of results.

You develop an understanding for planning the analytical process; data-related requirement handling, domain knowledge/modelling expertise and verification of results. Each student completes an industry cap-stone project as part of the course.

• Basic syntax and data types (None, bool, int, float, str, list, tuple, dict).
• Understanding the concept of variables and variable scope.
• Control structures (if-else, for and while loops).
• Ability to write simple functions and use them in programs.
• Basic Python libraries like math, random, as well as understanding and utilizing classes from the datetime module.

• Ability to handle basic string and list methods for manipulation. For example:

  For strings:

  • upper(): Converts all the characters in a string to uppercase.
  • lower(): Converts all the characters in a string to lowercase.
  • strip(): Removes leading and trailing whitespace from a string.
  • split(): Splits a string into a list where each word is a list item.
  • replace(): Replaces a specified phrase with another specified phrase.

  For lists:

  • append(): Adds an element at the end of the list.
  • insert(): Adds an element at the specified position.
  • remove(): Removes the first item with the specified value.
  • pop(): Removes the element at the specified position.
  • sort(): Sorts the list.
• Reading from and writing to files.
• Handling errors and exceptions using try/except blocks.
• An understanding of Python's logging system.

• Handle regular expressions for pattern matching in strings.
• Understand object-oriented programming: classes, objects, methods.
• Understand and use list comprehensions and lambda functions.
• Understand Python's memory management and optimization techniques.

Familiarity with the following Python libraries would be beneficial:

• NumPy: For numerical computations.
• Pandas: For data manipulation and analysis.
• Matplotlib: For data visualization.

Your code writing style should be clear, concise, and efficient. Avoid excessive code repetitions by adhering to the DRY (Don't Repeat Yourself) principle. This means that information is not duplicated, and you use appropriate structures to encapsulate repeated code. This not only reduces redundancy but also improves readability, maintainability, and scalability of the code. Code you produce should be self-explanatory, with meaningful variable, function, and class names. Comments should be used to explain the 'why' rather than the 'what' or 'how'. This promotes consistency and makes the code easier to understand and debug. Remember, code is read more often than it is written, so strive for clarity and simplicity.

Being comfortable with basic Linux terminal commands is essential to interact with Linux-based systems efficiently. These skills are indispensable for software and data engineers, whom our program is targeting. Knowledge of these terminal commands ensures a deeper understanding of computer systems and equips you with the tools needed for professional development in technology fields.

• man: Display the user manual of a command
• ssh: Secure shell remote login.
• ls: List directory contents.
• cd: Change the current directory.
• pwd: Print the name of the current directory
• cp: Copy files and directories.
• mv: Move or rename files and directories.
• rm: Remove files and directories.
• cat: Concatenate and display file content.
• echo: Display a line of text.
• head: Output the first part of files.
• tail: Output the last part of files.
• grep: Search for a specific pattern within files.
• find: Search for files in a directory hierarchy.
• chmod: Change the permissions of files or directories.
• chown: Change the owner and group of files or directories.
• df: Report file system disk space usage.
• du: Estimate file and directory space usage.
• tar: Archive files.
• gzip: Compress or expand files.
• ps: Report a snapshot of the current processes.
• top: Display Linux tasks.
• kill: Send a signal to a process.
• curl: Transfer data to or from a server.
• scp: Securely copy files between a local host and a remote host or between two remote hosts.
• rsync: Only transfers the changes made rather than transferring all the files again.

You should be able to use these Linux commands comfortably and understand their options and parameters. You should also understand the concept of Linux file permissions and know how to use pipes (|) and redirection (>, >>, <).