Data Architecture Principles

Do you know how well Data and Algorithms are used and treated in your organization?

Yes or No?

This page provides information on a set of 21 data architecture concepts and principles with which you can measure, analyze, improve, and optimize how data is used and treated in your company.

First, it explains what data concepts and data principles are, and what the benefits are of working with concepts and principles.

Next, a list of concepts with their principles is provided.

Then, you can download the dataset of data architecture principles, add your data to it, and start measuring how data and algorithms are used in your company.

Data Concepts

Data Concepts are approaches to work with data that is identified or exists outside your organization and could be implemented in your organization. Examples of data concepts are Zero Trust Data, Identity and Access Management, Data Sharing, and Data Validation.

Once a concept is made part of your data architecture, we call it a Data Architecture Concept.

The Current State Data Architecture of your organization is the set of implemented data concepts in your organization. If you manage data concepts as assets in a repository, you are in control over your data architecture.

Do you have a list of implemented data concepts?

Data Principles

A Data Principle is the way a data concept works or part of the data concept works. A concept can have many principles.

For example the Zero Trust Data Principle is: 'By providing no user on the network access to data before they are authenticated, it is ensured fewer people have unauthorized access to data, so fewer security incidents with data take place.'

The data principles tell you what the effect is of using data, treating data, storing, and retrieving data in a certain way.

Once, you have made a data concept part of your data architecture, the principle of that concept is a data architecture principle for your organization.

Most importantly, your implemented data architecture principles tell you how well you integrate data and algorithms in your business and let it flow uninterrupted as fuel through your processes and your systems. Managing principles gives you control over your data architecture.

Do you have a list of implemented data principles?

Algorithms

An algorithm is a powerful set of instructions for solving a problem or accomplishing a task.

Today, there are many developments and improvements in algorithms. Practice has shown that the more data you have, the better your algorithms work, and the more AI/Machine Language technologies you use, the better you can predict the future.

More and more IT systems or applications have embedded algorithms that work very efficiently to make optimal usage of data, AI, and ML, and predict the behavior of people very well.

But with great power comes great responsibility.

More and more organizations require reporting, managing, and controlling the algorithms they use. For this, we also can use data concepts and data principles.

An algorithm can be seen as a data concept doing all kinds of things with data. So, any algorithm in a company can be made visible and controllable by measuring the types of implemented data algorithms.

Algorithms are like tiny CRM/BI programs, qualifying and interpreting data. So, they contain variables, inputs, instructions, outputs, conditions, rules, and loops, we can look for.

An algorithm principle to recognize is Algorithm Data Qualification and Categorization principle: By establishing that person XXX lives in city AAA, has job BBB, education CCC and often buys DDD, it is ensured that person XXX falls into the category of target audience YYY.

Example Formula: YYY = XXX * (AAA && BBB && CCC && DDD).

Having Insight and Overview

So knowing which data architecture principles are implemented (and how well) in your organization, reveals a lot of information you can use to innovate and compete better.

The question is also which data concepts and principles you need to have implemented at which maturity level because of your strategy and business model. This is all about the future state data architecture.

The better your data architecture (ie. concepts and their principles) is aligned with your strategy and business model, the better you can execute your strategy and run your business model.

The benefit of working with Concepts and Architecture Principles

A concept (an abstraction of implementation or approach) always has one or more principles (the way the concept works, producing results).

Scientists discover and develop new concepts and principles every day. They help you to innovate and compete.

Knowing the principle of a concept, helps you decide whether you need the concept for your company or not because it teaches you which results are produced.

List of Data Architecture Principles

The following data architecture principles help you improve your data architecture and thus your organization's strength.

First, the concept is named, then the first principle of the concept is stated. Where possible a literature reference and a (design)guideline are provided.

Concept	Principle	Reference	Guidelines
1. Data (Asset) Management	By ingesting, storing, organizing, and maintaining the data created and collected by the organization via documented and mature processes it is ensured more-informed business decisions, improved marketing campaigns, optimized business operations and reduced costs can be made and with that increasing revenue, profits, existence and business continuity.	n/a	...
2. Data Validation	By validating all data at the point of entry, it is ensured that the quality of the data in the system is increased.	n/a	...
3. Data Discovery	By automating regular data discoveries, it is ensured that the organization knows how much data it is getting in, which data sets are aligned, and which applications need to be updated.	n/a	...
4. Data Sharing	By sharing data with other departments, it is ensured that silos in the organization are removed and more people have a 360 client view.	n/a	...
5. Optimal Interfaces	By providing the right interfaces to users, it is ensured that data can be easily shared and is accessible to others.	n/a	...
6. Data Security and Access Control	By developing access policies and data access controls at the raw data level, data is much more secured and access is controlled better.	n/a	...
7. Data Privacy	...	n/a	...
8. Common Vocabulary	By establishing a common vocabulary it is ensured that consistency is realized.	n/a	...
9. Data Curation	By curating data (like modeling the correct data relationships and cleansing data), it is ensured that the perceived and actual data quality is increased.	n/a	...
10. Data Integration	By logically integrating data, it is ensured that less data is copied for completeness of data view.	n/a	...
11. Data Elimination	By eliminating data copies and movement of data, it is ensured that costs are lower, the quality of data higher, and the organization is more agile.	n/a	...
12. Data Analyses/Intelligence	...	n/a	...
13. Data Algorithm Qualification and Classification	...	n/a	...
14. Data Prediction	...	n/a	...
15. Data Visualization	...	n/a	...
16. Data Lake	...	n/a	...
17. Data Warehouse	...	n/a	...
18. Data Virtualization	...	n/a	...
19. Data Hub	...	n/a	...
20. Data Complexity	...	n/a	...
21. Data Transactions	...	n/a	...
22. Zero Trust Data	By providing no user on the network access to data before they are authenticated, it is ensured fewer people have unauthorized access to data, so fewer security incidents with data take place	n/a
23. Search Query Logging	[By] having applications and databases always log the search queries in detail from users (preferably through a generic service), [it is ensured that] fraudulent and other malicious activities can be recognized and discovered better and faster, [thus] increasing the security, reliability, and stability of the environment and reducing risks and costs. (proposed search query logging details: datetime, username, userrole, context:process/task/application/feature and search text)	n/a

Are some of these data principles of interest to you? Give it a thought for a moment!

Download Dataset

The above list is available as an open data architecture principles set (in JSON format).

Visit the Datasets page.

Here you can download the dataset data architecture principles

Next, you can upload the data to watch it in the Dragon1 Viewer.

Understanding Data Sharing Principles

Many organizations want to break down their silos by sharing data across departments.

But 1 or 2 years after they have decided there are still many silos in the organization. Various departments are not sharing their data that makes sense to share, when in fact they could.

NOTE: In business, organizational silos refer to departments that operate independently from other departments and are not sharing data with others.

Suppose the organization has said, 'Data, metadata, products, and information from one business division, should be fully and openly shared with other business divisions, subject to national or international jurisdictional laws and policies, including respecting appropriate extant restrictions and under international standards of ethical research conduct.'

According to the Dragon1 method this statement is not favored to be labeled as a principle but as a general rule or guideline. The rationale behind this general rule or guideline will often contain what Dragon1 favors as the principle.

Why does Dragon1 not label this statement above as a principle?

If we consider Data Sharing as a concept, then we can describe the way the concept of Data Sharing works and realizes outcomes. We can describe how key elements collaborate and produce results.

We can focus on what is always true concerning how things work.

When we stumble upon these things, then we are describing the principles of Data Sharing.

A principle of Data Sharing could be 'By identifying all data that could be shared from any division and has clear value for that, and by removing all obstacles for sharing and making sharing mandatory via policies, it is ensured that more data is shared between business divisions and that silos are broken down.'

The description above is a way of working that is always true (or at least highly likely to be always true). It is a working mechanism. (of course, this example has to be researched more.)

Once, we have this knowledge of a principle (on how the world works), it will influence how we design solutions, systems, and policies.

So, the principle above has an impact.

For instance, the key elements of a data sharing policy often are missing, or a list of identified data that makes sense to share. And if you implement the key elements, you increase the chances of actually sharing data and breaking down silos.

If you like the above, please try it in your work.

If you don't like it, just keep using and relying on your current habit.

What To Do - Checklist

Here follows a checklist on what best to do with the principles:

1. Make an inventory of data concepts and data architecture principles that are currently implemented in your company.
2. Use the provided list of principles here as a reference or starting point.
3. Collect the business process flow diagrams (BPMN), data diagrams (DMN) application components diagrams (UML and ArchiMate) that and IT Infrastructure diagrams (Azure, Amazone, Citrix, or IBM models) are or should be affected by the data architecture principles.
4. Identify how processes, data, applications, and IT infrastructure are or should have been affected by the principles.
5. Analyze the gap.
6. Create a roadmap to fill that gap.

Measure, Visualize and Rationalize

It is important for any organization to measure how well data architecture principles are implemented and at which maturity level.

Also, it is important to rationalize which data architecture principles one needs and does not need.

This is all-important because it makes you sit in the driver's seat of the strategy of the organization.

Data, in many organizations, will soon be an uncontrollable complex whole.

The better you control or manage your data or its complexity, the better you can compete.

Get Started

The above steps are supported on the Dragon1 platform. Create an account and get guided to document, measure, rationalize, and improve your data architecture principles.