Skip to content
Heterodata An Arcanum Research project Shaikh Data
Shaikh Data
The comparative layer
All datasets Wealth of Nations (Shaikh & Tonak 1994) Capitalism (Shaikh 2016) Comparative studies
GitHub ↗

Methodology & provenance

How The comparative layer is constructed, where every number comes from, and what is and isn’t covered. This is the dataset-level Data Provenance Record; each series also carries its own DPR in the Explore construction drawer.

Explore the data → See the pipeline → GitHub ↗

Data Provenance Record

Project
RED — the comparative scholarship layer
Producer
Arcanum Research
Summary
Where scholars have measured the same Marxian categories from the same US accounts and reached different numbers, this layer puts those measurements beside one another instead of letting one stand in for the rest. It carries the net social wage as four separate lines (Shaikh & Tonak's book line, the 2000 revision, Tonak's 2025 update and Moos's 2019 replication), Mohun's own published recomputation of the classification, and the source-vintage dossiers that explain why the underlying BEA data moved. Every value is a digitization of what its author actually printed or stored, attributed to that author and resolved to a full bibliographic record.
Attribution
Cite the named primary source in each series' provenance record; cite the reconstruction as the project (CITATION.cff ships in every download bundle).
Coverage
135 of 136 series carry plottable data, organised into 3 groups
Construction
Built with the Anu data-construction pipeline. Each script carries a phase prefix — Loading (L), Processing (P), Validation (V), Manual adjustment (M), Analysis (A), Output (O) — so the build reads as an ordered method. No synthetic, placeholder, or proxy values; series that cannot be sourced are marked data-unavailable.
Provenance records
Each series ships a full provenance record, downloadable from the explainer on its Explore page.
Units
Every series carries an explicit unit (per subseries where they differ); no mixed-unit columns. Units travel with the series in the download files and the data dictionary.
Downloads
CSV + XLSX + Parquet (zipped) · per-series download catalogue
Source code & data
https://github.com/andenick/measuring-the-wealth-of-nations
Refresh cadence
Refreshed on demand (no fixed schedule) — manual RED rebuild + web re-export when a comparative lane is revised or a new study is added

Data vintage last built 2026-08-21 · refreshed on demand (no fixed schedule).

How each series is built

  1. L##

    Loading

    Fetch and read the raw source data — API pulls, archived tables, digitised book figures — and validate that the fetched units match what the named source promises.

  2. P##

    Processing

    Construct each series: rebase, splice vintages, deflate, and combine inputs. Composite and formula series are assembled here, with a dimensional-analysis check whenever units differ. Processing — and processing only.

  3. V##

    Validation

    Check every constructed series against the original published figures and tables — mean absolute error, endpoint sanity, unit and scale audits. A series only passes when it reproduces the source values. The exact figure, table, or page being reproduced is cited in that series' provenance record (Explore → the construction drawer).

  4. M##

    Manual adjustment

    Apply documented hand corrections where the source itself requires them — each adjustment recorded with its rationale, never a silent edit.

  5. A##

    Analysis

    Derive the downstream analytical quantities the book reports (ratios, trends, decompositions) from the validated series.

  6. O##

    Output

    Write the publishable artifacts: the tidy per-series CSVs, the Parquet files, the data dictionary, and the per-series provenance records that ship with the data.

Full project documentation

RED — Real Econ Data: web export v1.1.0

A public, download-ready projection of the RED data project. Nothing here is recomputed: every value is carried over unchanged from the constructed series.

What is in it

Slice Contents Published series Withheld
mwn/ Shaikh & Tonak, Measuring the Wealth of Nations (1994) 59 4
ccc/ Shaikh, Capitalism: Competition, Conflict, Crises (2016) 110 6
comparative/NSW/ Net social wage: the competing lines compared 117 —
comparative/MOHUN/ Mohun's recomputation of the categories 19 —
comparative/BEA_VINTAGE/ Source-vintage and classification dossiers 2 dossiers —

How to use it

  • WEB_MANIFEST.json (root) is the index: slices, counts, and everything withheld.
  • Each slice has its own WEB_MANIFEST.json, series_registry.json and data_dictionary.csv.
  • Each comparative section has a SECTION_MANIFEST.json listing every component that can be downloaded on its own.
  • data_dictionary.csv (root) is one row per component across all slices, each with a full source string; CITATION_REGISTRY.json holds the bibliographic records those rows point at.

Formats

Every series ships as CSV and Parquet. The net-social-wage comparison also ships as an Excel workbook, and each comparative section ships its study as PDF. The two book modules keep the layouts they were built in: mwn/ is wide (a metadata row, a component-ID row, then one row per year), ccc/ and the comparative layer are long (year, value, subseries_id, source_id, units).

What is withheld, and why

Nothing is dropped silently. Every withheld series is listed with its reason in WEB_MANIFEST.json and in its slice registry. Two kinds of exclusion occur: series triaged as not sensibly publishable (a matrix reduced to a scalar, a single-year survey snapshot, a series whose construction was found unfaithful), and one series withheld for licensing — its book-period source is commercially licensed and may not be redistributed.

Series that were removed from the project outright, because their claimed provenance could not be substantiated, are named under de_registered_series in WEB_MANIFEST.json. None of their values appear anywhere in this export.

Citations: 277 bibliographic records ship; 1 are excluded (incomplete records, and the licensed source above), each listed with a reason.

Version

v1.1.0 (2026-08-19): full regeneration from the corrected sources after the zero-fabrication purge and the RM1-RM6 remediation campaign. The v1.0.0 export had only been patched in place and its metadata layer still asserted pre-purge counts. The directory name is deliberately unchanged at red_Web_v1.0.0 so that existing dataset configurations and download paths keep resolving; the version of the export is carried in this field, not in the directory name.