Full Course: 🏗️ Big Data and Machine Learning in Construction.

May 31, 2020

Category. Big Data, Construction, Machine Learning
Tag. Big data, Machine Learning

🛴 Practical Step-by-Step Course for Beginners. In this course, we will step by step, using the example of real data, we will go through the main processes related to the topic “Big Data and Machine Learning in Construction”.

📚 You will be guided through the basics of using Python, Kaggle, GitHub, Spark, MySQL, Jupyter Notebook. By following along with provided code, you will experience how one can perform predictive modeling and leverage graph analytics to model problems. This specialization will prepare you to ask the right questions about data and do basic exploration of large, complex datasets.

20% Off for all Сourses. Unlimited for all new Students:

🎀 Coupon: Coupon20getoff

🙋🏼‍♂️ If you want to try or get even more coupons – please write to me.

📚 Course content:

Since the material turned out to be voluminous, I divided the course into five parts.

💻 Part 1. Collect Data. Extract Data from PDF. PDF to Excel

The first part is devoted to the collection and extraction of data from PDF documents. (⏱ 1 hour, 📚10 lessons).

Big Data. Extract Data from PDF Drawings and Documents. PDF – Excel – Charts and Graphs. 📚 10 Lessons

✔️ In this course, you will learn how to extract data from PDF documents, drawings and any other documents in PDF format. We will have two sets of data consisting of PDF files that we will transform to the text and to tabular form. We will visualize the received data on the Kaggle platform using python libraries, which will help us to depict our received data in a graphical format.

✔️ During the training process, we will install Python and such libraries as Pandas, seaborn, matplotlib and others. We will upload the received data to the Kaggle platform and here using the “Jupiter Notebook” we will visualize our data and at the end, we will upload our data to the GitHub platform.

💻 Part 2. Convert Scanned Documents to Text. JPG to Excel

The second part is devoted to the collection and extraction of data from scanned documents and Images (⏱1.20 hour, 📚10 lessons).

Collect Data. Convert Scanned Documents to Text. Extracting Data from Contracts and Receipts. 📚 10 Lessons

✔️ In this course, you will learn how to extract data from From Scanned Documents And Images, invoices, receipts, contracts and any other documents in PDF format or in Image format.We will work on real data. We will have two sets of data consisting of PDF files that we will transform to the text and to tabular form. We will visualize the received data on the Kaggle platform using python libraries, which will help us to depict our received data in a graphical format.

✔️ During the training process, we will install Python and such libraries as Pandas, seaborn, matplotlib and others. We will upload the received data to the Kaggle platform and here using the “Jupiter Notebook” we will visualize our data and at the end, we will upload our data to the GitHub platform.

💻 Part 3. Store and Process Big Data. Excel, MySQL, Spark

In third part we will consider the main options for storing big data. (⏱50 minutes, 📚9 lessons)

Store and Process Project Data and Big Data. From Excel to MySQL and Spark. 📚 8 Lessons

✔️ In practical lesson we will install the MySQL server on computer and learn how to work and edit MySQL databases.In the fifth lesson we will take one regular Excel table and transfer the information from this table to the MySql server.

✔️ Then we will install the spark in order to work with datasets in a distributed manner. Then, to process the distributed data, we export the data from MySQL into spark. And with the help of Jupiter Notebook, we prepare the data for visualization of this data.

💻 Part 4. Big Data Visualization and Visualization Tools

In fourth part we will look at the main platforms for visualizing Big Data and consider the main Data Visualization Online-Tools for Big Data (⏱ 50 minutes, 📚8 lessons).

Visualization of Project Data in Construction. Basic Data Visualization Tools. 📚 8 Lessons

✔️ We will briefly look at these platforms and generate several reports in each of the platforms. This will give you the opportunity to choose the right platform that suits you and your data.

✔️ In practical lesson we exported an excel file with our data to the Kaggle platform and using a Jupyter Notebook we cleared the data and visualized the data using different python libraries.

💻Part 5. Machine Learning. Price and Time Prediction

In fifth part we will examine in detail the basic types, terms and algorithms of machine learning. We go through the basic concepts of machine learning that beginners need. We will consider in more detail such algorithms as K-means supervised Machine Learning, Linear Regression and other algorithms for Machine Learning.

Machine Learning in Construction. Predicting the Time and Cost of Projects . 📚 9 Lessons

✔️ In practical lessons we will predict the time and cost of construction for the new project X, based on the data that we collected on previous projects. And in another lesson we will predict the cost of building project X and construction time by the parameters that we will set for the new project x

✔️ Then we take open source data for the San Francisco city. We will clear this raw data and display the data in the form of a charts and maps. We will collect various interesting insights from this public information. Then we will prepare the data to create a machine learning model and try to predict some parameters from this data.

Requirements and Target Audience

💾 I recommend repeating the steps in this course using your own documents and spreadsheets. If you do not have them, you can use the files and documents that you will find in the attachment to the relevant lessons.