Skip to content
Heterodata An Arcanum Research project Shaikh Data
Shaikh Data
Shaikh Capitalism Data GitHub ↗

Methodology & provenance

How Shaikh Capitalism Data is constructed, where every number comes from, and what is and isn’t covered. This is the dataset-level Data Provenance Record; each series also carries its own DPR in the Explore construction drawer.

Data Provenance Record

Project
Replication and Extension of Shaikh (2016) Capitalism: Competition, Conflict, Crises
Producer
Arcanum Research
Summary
An open replication and historical extension of the empirical material in Anwar Shaikh's Capitalism: Competition, Conflict, Crises (Oxford, 2016). 110 published series across the book's chapters, the book appendix, and the Sraffa-price follow-up studies — each chart traceable to a named source, a construction method, a provenance record and a full bibliographic citation.
Attribution
Cite the named primary source in each series' provenance record; cite the reconstruction as the project (CITATION.cff ships in every download bundle).
Period covered
1860–2010 (replication core); 2011–2025 (modern extension)
Coverage
110 of 110 series carry plottable data, organised into 16 groups
Construction
Built with the Anu data-construction pipeline. Each script carries a phase prefix — Loading (L), Processing (P), Validation (V), Manual adjustment (M), Analysis (A), Output (O) — so the build reads as an ordered method. No synthetic, placeholder, or proxy values; series that cannot be sourced are marked data-unavailable.
Provenance records
Each series ships a full provenance record, downloadable from the explainer on its Explore page.
Units
Every series carries an explicit unit (per subseries where they differ); no mixed-unit columns. Units travel with the series in the download files and the data dictionary.
Downloads
CSV + XLSX + Parquet (zipped) · per-series download catalogue
Source code & data
https://github.com/andenick/shaikh-capitalism-data
Refresh cadence
Refreshed on demand (no fixed schedule) — manual RED rebuild + web re-export when source data is revised or the modern extension is updated

How each series is built

  1. L##

    Loading

    Fetch and read the raw source data — API pulls, archived tables, digitised book figures — and validate that the fetched units match what the named source promises.

  2. P##

    Processing

    Construct each series: rebase, splice vintages, deflate, and combine inputs. Composite and formula series are assembled here, with a dimensional-analysis check whenever units differ. Processing — and processing only.

  3. V##

    Validation

    Check every constructed series against the original published figures and tables — mean absolute error, endpoint sanity, unit and scale audits. A series only passes when it reproduces the source values. The exact figure, table, or page being reproduced is cited in that series' provenance record (Explore → the construction drawer).

  4. M##

    Manual adjustment

    Apply documented hand corrections where the source itself requires them — each adjustment recorded with its rationale, never a silent edit.

  5. A##

    Analysis

    Derive the downstream analytical quantities the book reports (ratios, trends, decompositions) from the validated series.

  6. O##

    Output

    Write the publishable artifacts: the tidy per-series CSVs, the Parquet files, the data dictionary, and the per-series provenance records that ship with the data.

Full project documentation

RED — Real Econ Data: web export v1.1.0

A public, download-ready projection of the RED data project. Nothing here is recomputed: every value is carried over unchanged from the constructed series.

What is in it

Slice Contents Published series Withheld
mwn/ Shaikh & Tonak, Measuring the Wealth of Nations (1994) 59 4
ccc/ Shaikh, Capitalism: Competition, Conflict, Crises (2016) 110 6
comparative/NSW/ Net social wage: the competing lines compared 117
comparative/MOHUN/ Mohun's recomputation of the categories 19
comparative/BEA_VINTAGE/ Source-vintage and classification dossiers 2 dossiers

How to use it

  • WEB_MANIFEST.json (root) is the index: slices, counts, and everything withheld.
  • Each slice has its own WEB_MANIFEST.json, series_registry.json and data_dictionary.csv.
  • Each comparative section has a SECTION_MANIFEST.json listing every component that can be downloaded on its own.
  • data_dictionary.csv (root) is one row per component across all slices, each with a full source string; CITATION_REGISTRY.json holds the bibliographic records those rows point at.

Formats

Every series ships as CSV and Parquet. The net-social-wage comparison also ships as an Excel workbook, and each comparative section ships its study as PDF. The two book modules keep the layouts they were built in: mwn/ is wide (a metadata row, a component-ID row, then one row per year), ccc/ and the comparative layer are long (year, value, subseries_id, source_id, units).

What is withheld, and why

Nothing is dropped silently. Every withheld series is listed with its reason in WEB_MANIFEST.json and in its slice registry. Two kinds of exclusion occur: series triaged as not sensibly publishable (a matrix reduced to a scalar, a single-year survey snapshot, a series whose construction was found unfaithful), and one series withheld for licensing — its book-period source is commercially licensed and may not be redistributed.

Series that were removed from the project outright, because their claimed provenance could not be substantiated, are named under de_registered_series in WEB_MANIFEST.json. None of their values appear anywhere in this export.

Citations: 277 bibliographic records ship; 1 are excluded (incomplete records, and the licensed source above), each listed with a reason.

Version

v1.1.0 (2026-08-19): full regeneration from the corrected sources after the zero-fabrication purge and the RM1-RM6 remediation campaign. The v1.0.0 export had only been patched in place and its metadata layer still asserted pre-purge counts. The directory name is deliberately unchanged at red_Web_v1.0.0 so that existing dataset configurations and download paths keep resolving; the version of the export is carried in this field, not in the directory name.