Data, Data-Management and Databases

Data

Regardless of the technology platform, targeted users or industries, all software application are designed to deal with data, and interact with different sources of data, including data from different sources, including database, Excel file, CSV file, JSON file, XML file, Web-API, Web-Services and etc.

Data security and Privacy

Data security and privacy are key concerns in today's application. In addition to preventing data breach, there are distinct legal mandates in different jurisdiction, which we must follow to avoid legal consequences.

Confidentiality, integrity and availability (CIA)

While the contents for this book is not about security, it's necessary for enterprise database application developer to have general understanding about CIA security concerns in database application, which is short for Confidentiality, Integrity and Availability.

Confidentiality

We need to ensure adequate access controls are implemented to prevent unauthorized access to data.

Integrity

We need to implement adequate protection to ensure data accuracy, consistency, and prevent unauthorized data tempering.

Availability

We need to implement adequate measures to keep data accessible by authorized entities.

Data management

In an large organization, there is generally a separate data management team, tasked with the responsibilities to maintain the organization's data, data storage infrastructure and controlling access to data. As a database programmer, you need to have general understanding about data management, including:

  • Database creation, backup and restore
  • Database queries and updates
  • Database access control
  • Managing database server access keys and secrets
  • Etc.

Data sources

Data and database are two separate subjects. There are different data sources, which may not be coming from a database such as:

  • CSV, JSON or XML files
  • Web-API or Web-Services
  • Data stream from IoT devices
  • Etc.

Databases

There are different type of databases, for different environments and purposes such as:

  • Microsoft SQL Server
  • Azure SQL
  • SQLite
  • Cosmos DB
  • MySQL
  • PostgreSQL
  • Oracle
  • Access
  • Etc.