The decision to use Jupyter Notebook for results verification turned out to be highly beneficial. Our experience with solutions from Data Driven Construction and Jupyter Notebook has been extremely positive.
Overall, the integration of solutions from Data Driven Construction and Jupyter Notebook has greatly improved our workflows. Their combined capabilities and performance have not only optimized data processing, but also produced meaningful results, making them highly recommended tools.
Change language
Post's Highlights
Stay updated: news and insights
We’re Here to Help
Fresh solutions are released through our social channels
Dive into the world of data-driven construction with this accessible guide, perfect for professionals and novices alike.
From the basics of data management to cutting-edge trends in digital transformation, this book
will be your comprehensive guide to using data in the construction industry.
About 10,000 years ago, in the Neolithic era, mankind made a revolutionary transition in its development, abandoning the nomadic lifestyle in favor of sedentary life, which led to the appearance of the first primitive buildings...
The first documentary evidence in construction dates back to the period of pyramid building, around 3000-4000 BC (“Papyrus, 3rd century B.C. Language is Greek,” 2024). Since then, the keeping of written records has facilitated and...
At the heart of any process is the transformation of past experience into a tool for planning the future. Experience in the modern sense is a structured set of data, the analysis of which allows...
For millennia, the amount of information recorded in construction has barely changed, but it has grown rapidly in recent decades (Fig. 1.1-5). According to the PwC study® “Managed Data. What Students Need to Succeed in...
The era of modern digital data storage and processing began with the advent of magnetic tape in the 1950s, which opened up the possibility of storing and utilizing large amounts of information. The next breakthrough...
Today’s companies are faced with the need to integrate multiple data management systems. Selecting data management systems, managing these systems well, and integrating disparate data sources is becoming critical to business performance. In the mid-2020s,...
The process of integrating data into applications and databases relies on the aggregation of information from a variety of sources, including different departments and specialists (Fig. 1.2-4). Specialists search for relevant data, process it, and...
The construction industry is experiencing an unprecedented information explosion. If we think of business as a knowledge tree (Fig. 1.2-5) fed by data, the current stage of digitalization can be compared to the rapid growth...
In the last two years, 90% of all existing data in the world has been created (B. Marr, “How much data do we create every day? The Mind-Blowing Stats Everyone Should Read,” 2018). As of...
In recent years, more and more companies are outsourcing data storage to cloud services. For example, if a company hosts half of its data in the cloud, at an average price of $0.015 per gigabyte...
The evolution of data in construction is a journey from clay tablets to modern modular platforms. The challenge today is not to collect information, but to create a framework that turns disparate and diverse data...
Whether it is large corporations or medium-sized companies, specialists are daily engaged in filling program systems and databases with various interfaces with multiformat information (Fig. 3.2-1), which, with the help of managers, must interact with...
Today, most companies are facing a paradox: about 80% of their daily processes still rely on classic structured data – familiar Excel spreadsheets and relational databases (RDBMS) (М. Shacklett, “Structured and unstructured data: Key differences,”...
Data in information systems are organized in different ways – depending on the tasks and requirements for storing, processing and transmitting information. The key difference between the types of data models, the form in which...
One of the key challenges faced by construction companies during digitalization is limited access to data. This makes it difficult to integrate systems, reduces the quality of information and complicates the organization of efficient processes....
The construction industry was one of the last to address the problem of closed and proprietary data. Unlike other sectors of the economy, digitalization has been slow to develop here. The reasons for this include...
The construction industry is undergoing a shift that cannot be monetized in the usual way. The concept of data-driven, data-centric approach and the use of Open Source tools is leading to a rethinking of the...
While in past decades business sustainability was largely determined by the choice of software solutions and dependence on specific vendors, in today’s digital economy the key factor is data quality and the ability to work...
The emergence of Large Language Models (LLMs) was a natural extension of the movement towards structured open data and the Open Source philosophy. When data becomes organized, accessible and machine-readable, the next step is a...
Big language models (ChatGPT, LlaMa, Mistral, Claude, DeepSeek, QWEN, Grok) are neural networks trained on huge amounts of textual data from the Internet, books, articles and other sources. Their main task is to understand the...
The appearance of the first chat-LLMs in 2022 marked a new stage in the development of artificial intelligence. However, immediately after the widespread adoption of these models, a legitimate question arose: how secure is it...
Modern tools allow companies to deploy a large language model (LLM) locally in just a few hours. This gives complete control over data and infrastructure, eliminating dependence on external cloud services and minimizing the risk...
The next stage in the evolution of LLM application in business is the integration of models with actual real-time corporate data. This approach is called RAG (Retrieval-Augmented Generation) – Retrieval-Augmented Generation. In this architecture, the...
When diving into the world of automation, data analysis, and artificial intelligence – especially when working with large language models (LLMs) – it is critical to choose the right integrated development environment (IDE). This IDE...
Pandas occupies a special place in the world of data analysis and automation. It is one of the most popular and widely used libraries of the Python programming language(“Python Packages Download Stats,” 2024), designed to...
DataFrame is the central structure in the Pandas library, which is a two-dimensional table (Fig. 3.4-6) where rows correspond to individual objects or records and columns correspond to their characteristics, parameters, or categories. This structure...
In this part, we reviewed the key types of data used in the construction industry, got acquainted with different formats of their storage and analyzed the role of modern tools, including LLM and IDEs, in...
In the era of the data-driven economy, data is becoming the basis for decision-making rather than an obstacle. Instead of constantly adapting information to each new system and its formats, companies are increasingly striving to...
One of the most common tasks in construction projects is to process specifications in PDF format. To demonstrate the transition from unstructured data to a structured format, let’s consider a practical example: extracting a table...
In addition to PDF documents with tables (Fig. 4.1-2) and scanned versions of tabular forms (Fig. 4.1-5), a significant part of information in project documentation is presented in text form. It can be both coherent...
Structuring and categorizing CAD data (BIM) is more challenging because data stored from CAD (BIM) databases are almost always in closed or complex parametric formats, often combining geometric data elements (semi-structured) and metainformation elements (semi-structured...
From 2024, the design and construction industry is undergoing a significant technological shift in the use and processing of data. Instead of free access to design data, CAD -system vendors are focusing on promoting the...
Today’s design data architecture is undergoing fundamental changes. The industry is moving away from bulky, isolated models and closed formats towards more flexible, machine-readable structures focused on analytics, integration and process automation. However, the transition...
Effective data management requires a clear standardization strategy. Only with clear requirements for data structure and quality can data validation be automated, manual operations reduced and informed decision making accelerated at all stages of a...
As the number of digital systems within companies grows, so does the need for data consistency between them. Managers responsible for different IT systems often find themselves unable to keep up with the increasing volume...
In the context of digitalization and automation of inspection and processing processes, a special role is played by classification systems elements – a kind of “digital dictionaries” that ensure uniformity in the description and parameterization...
Historically, construction element and work classifiers have evolved in three generations, each reflecting the level of available technology and the current needs of the industry in a particular time period (Fig. 4.2-8): First generation (early...
Effective management of data (structured and categorized by us earlier) is impossible without a well thought-out storage and processing structure. To ensure access and consistency of information at the storage and processing stages, companies use...
Having a data model and description of entities through parameters, we are ready to create databases – storages, where we will store information coming after the structuring stage on specific processes. Let’s try to create...
With data becoming one of the key strategic assets, companies need to do more than just collect and store information correctly – it is important to learn how to manage data systematically. The Center of...
This website uses cookies
We use cookies to personalise content and ads, to provide social media features and to analyse our traffic. We also share information about your use of our site with our social media, advertising and analytics partners who may combine it with other information that you’ve provided to them or that they’ve collected from your use of their services.
Cookies are small text files that can be used by websites to make a user's experience more efficient.
The law states that we can store cookies on your device if they are strictly necessary for the operation of this site. For all other types of cookies we need your permission. This means that cookies which are categorized as necessary, are processed based on GDPR Art. 6 (1) (f). All other cookies, meaning those from the categories preferences and marketing, are processed based on GDPR Art. 6 (1) (a) GDPR.
This site uses different types of cookies. Some cookies are placed by third party services that appear on our pages.
You can at any time change or withdraw your consent from the Cookie Declaration on our website.
Learn more about who we are, how you can contact us and how we process personal data in our Privacy Policy.
Please state your consent ID and date when you contact us regarding your consent.
Linux
macOS
🚀 Welcome to the future of data in construction!
You're taking your first step into the world of open data, working with normalized, structured data—the foundation of data analytics and modern automation tools.
By downloading, you agree to the DataDrivenConstruction terms of use
Stay ahead with the latest updates on converters, tools, AI, LLM
and data analytics in construction — Subscribe now!
🚀 Welcome to the future of data in construction!
You're taking your first step into the world of open data, working with normalized, structured data—the foundation of data analytics and modern automation tools.
By downloading, you agree to the DataDrivenConstruction terms of use
Stay ahead with the latest updates on converters, tools, AI, LLM
and data analytics in construction — Subscribe now!
🚀 Welcome to the future of data in construction!
You're taking your first step into the world of open data, working with normalized, structured data—the foundation of data analytics and modern automation tools.
By downloading, you agree to the DataDrivenConstruction terms of use
Stay ahead with the latest updates on converters, tools, AI, LLM
and data analytics in construction — Subscribe now!
🚀 Welcome to the future of data in construction!
You're taking your first step into the world of open data, working with normalized, structured data—the foundation of data analytics and modern automation tools.
By downloading, you agree to the DataDrivenConstruction terms of use
Stay ahead with the latest updates on converters, tools, AI, LLM
and data analytics in construction — Subscribe now!
🚀 Welcome to the future of data in construction!
You're taking your first step into the world of open data, working with normalized, structured data—the foundation of data analytics and modern automation tools.
By downloading, you agree to the DDC terms of use
🚀 Welcome to the future of data in construction!
You're taking your first step into the world of open data, working with normalized, structured data—the foundation of data analytics and modern automation tools.
By downloading, you agree to the DataDrivenConstruction terms of use
Stay ahead with the latest updates on converters, tools, AI, LLM
and data analytics in construction — Subscribe now!
🚀 Welcome to the future of data in construction!
By downloading, you agree to the DataDrivenConstruction terms of use
Stay ahead with the latest updates on converters, tools, AI, LLM
and data analytics in construction — Subscribe now!
CHAPTER 016: HiPPO or the Danger of Opinions in Decision Making
Discusses the importance of structured data over subjective opinions for decision-making.
CHAPTER 035: Data Transformation: The Critical Foundation of Modern Business Analysis
Covers data cleaning, validation, and structuring, including handling "dirty" data.
CHAPTER 025: Structured Data
Explores working with structured formats like XLSX, CSV, and SQL.
CHAPTER 051: Learning how to turn documents, PDF, pictures and texts into structured formats
Practical methods for converting different Documents abd geometric data into structured tables.
CHAPTER 007: Corporate Mycelium: How Data Connects to Business Processes
Explains data integration across systems via "mycelium" networks of managers and tools.
CHAPTER 014: Data Silos and Their Impact on Company Performance
Challenges of siloed systems and the need for workflow automation.
CHAPTER 132: n8n & Workflow Automation
Examples of integrating data using ETL tools (Apache NiFi, Airflow, n8n).
CHAPTER 046: Choosing an IDE: From LLM Experiments to Business Solutions
Setting up automated pipelines in Jupyter Notebook, VS Code, and Google Collab.
CHAPTER 041: LLM Chatbots: ChatGPT, LlaMa, Mistral, Claude, DeepSeek, QWEN, Grok for Automating Data Processing
The role of LLMs in data analysis and code generation.
CHAPTER 045: RAG: Intelligent LLM Assistants with Access to Corporate Data
Retrieval-Augmented Generation (RAG) for document and database queries.
CHAPTER 044: AI in the Company and How to Deploy Your Own LLM
implementing local LLMs to process sensitive construction data
CHAPTER 029: Text Data: Between Unstructured Chaos and Structure
details methods to transform raw text (contracts, reports) into structured formats
