Data analysts need access to structured data. Open data formats give analysts the best chance of drawing conclusions about data.

During the course of data warehousing operations, data should be cleaned into a standard format. A standard format is one that will guarantee the highest chance of interoperability with past and future data sets. Through the combination of data sets of high quality, relevance, and accuracy we may draw conclusions valid to an organization.

tl;dr: recommended file formats are SQLite, JSON. Recommended analysis tools are DuckDB and jupyter notebooks.

Relational data

Most data today is stored in relational databases.

all data lives in tables; tables have columns, and rows are addressable with keys; C.R.U.D.; schemas; a textual language to convey these concepts. The language, of course, is SQL¹

SQLite is a database file format that is the perfect way to store datasets. It is:

1. Structured: data that is stored in SQLite must conform to a tabular format that can be easily manipulated
2. Typed: Each cell of data must conform to a specific data type, or else be explicitly marked as TEXT or a binary BLOB.
3. Maintained: SQLite's on-disk file format and C-language API are fully backwards compatiable, documented, and supported until 2050²
4. Portable: SQLite files are identical bit-for-bit on all platforms²

References