PipeWhy Enron source contract E1 version1.0 | PW-14.01-.07 | Q1 v1.0
November2,2026

Accepted source and scope (D1, PW-14.01)

The user selected https://huggingface.co/datasets/corbt/enron-emails.
Initial scope is its complete default/train split:517401 records in three Parquet
shards,493482449 compressed bytes. This is a public dataset connector, not an
enterprise mail-server connector or permission to send email. First pinned revision:
cfc06c758093d90993abce1a43668fb7357258a6. ENRON_SOURCE_MANIFEST.json records the
actual repository/card/Parquet audit and published LFS SHA256 identities.
Both Windows CPython3.14 and Linux source/distribution gates remain applicable;
actual full-source qualification must be recorded separately from synthetic tests.

Consistency and history (PW-14.02/.03, D5 implementation scope)

Require an explicit full40-character Git revision; never resolve latest during a
scan or silently substitute another version. Resolve metadata and shard URLs at
that same revision. Verify all declared byte counts/SHA256 hashes before parsing.
Use per-revision local cache directories, temporary files and atomic publication;
an interrupted/mismatched download must not become a verified cached file. Compare
existing pinned metadata rather than overwriting conflicting identities.

A read uses verified file handles and checks file identity/content stability; a
concurrent replacement/mutation must fail, not publish a complete comparable
observation. Readiness is not source consistency. Captures retain the exact Git
revision, schema/profile/context and successful/failed read counts. A bounded
reader completes only after the declared record count is consumed successfully.

Historical reads require either the same accessible upstream revision or its
verified local cache. Offline reads never contact upstream or fall back to head.
No guarantee is made that Hugging Face retains any revision forever. The connector
does not evict caches automatically; explicit removal makes missing cached history
unavailable. C5 snapshot history is independent: captured observations remain
readable without network/Parquet libraries, and retained C2 values are opt-in.
No enterprise retention/storage budget or D8/D9 deployment choice is inferred.

Initial D5 implementation defaults: admit at most512 MiB of compressed source
files per explicitly requested revision and4 MiB of metadata per response. The
audited493482449-byte revision fits this cap. No background version prefetch or
automatic eviction is performed. Multiple retained revisions require explicit
caller requests and caller-managed disk retention. Dataset-level C5 fingerprints
are the default; retaining email-value frames is an explicit option subject to
existing snapshot limits. These are bounded local integration defaults, not an
accepted enterprise SLA or a promise that source history remains available.

Type and record mapping (PW-14.04)

Source field order/IDs are fixed:
  0 message_id: nullable string
  1 subject: nullable string
  2 from: nullable string
  3 to: nullable ordered list of nullable strings
  4 cc: nullable ordered list of nullable strings
  5 bcc: nullable ordered list of nullable strings
  6 date: nullable timestamp[us,tz=UTC]
  7 body: nullable string
  8 file_name: nullable string

Validate the exact nine-column Arrow schema before reading. Strings remain exact
Unicode text; nulls remain C2 nulls. Recipient lists map to versioned canonical JSON
text (UTF8, ensure_ascii=False, compact separators), preserving ordering, duplicate
entries, empty strings, null elements, empty lists and null whole fields distinctly.
Do not join/filter/sort addresses. This is an explicit list representation in the
source schema identity, not a new scalar type or a claim that text equals a list.
The existing strict profile remains default; logical scalar normalization is opt-in.

Convert UTC microseconds exactly to timezone-aware UTC datetime for existing C2
timestamp normalization; do not pass through floating-point seconds or reinterpret
mail dates as observation/operation times. Preserve published typed data, not a
claim of byte identity with the original CMU/raw-mail archive. Email bodies and
headers are data; no instructions, links or apparent credentials are executed.

Emit existing flat C2 RecordBatch buffers with all nine fields per record, preserving
duplicate multiplicity. Bound both Arrow row batches and emitted C2 record/word
pools; reject a single over-limit record explicitly rather than truncating it.
Do not materialize the full corpus as Python dicts or install pandas/datasets just
to read Parquet. PyArrow26.0.0 is the adopted optional enron reader; CPython3.14
Windows/Linux wheels were verified. ENRON_ACCEPTANCE.txt records production
unit/type/coverage, dependency, package and full-source qualification evidence.

Identity/discovery (PW-14.05/.06)

Namespace identifies the selected repository/split, independent of download paths.
Artifact0 is the repository container; artifact1 is the logical train dataset with
parent0. Physical Parquet shard names/order are storage details, not artifact IDs.
Display labels are separate from these fixed IDs. Initial discovery supports root0
and dataset1, with bounded C1 batches and explicit unknown-scope errors. No mailbox
or individual-message artifact catalogue is claimed by this first scope.

file_name is the published record key; message_id is content, not an assumed global
deduplication key. Current full-source audit finds both unique, but duplicate
content still counts. Row position is only the global ordered position. A record
key rename is a content change; record-level rename tracking is not claimed.
Lexicographic pinned shard paths and physical row order define ordered iteration;
multiset fingerprints remain insensitive to partition/order. Scope selection never
silently advertises a truncated prefix as the full train split.

The selected repository is public/ungated. Configuration loading never connects;
network preparation is an explicit operation. No token is required or read from
ambient credentials. HTTP/permission/offline/missing/corrupt/type failures remain
actionable and cannot yield an empty complete dataset. Metadata pagination or
unrecognized layouts must be exhausted/validated or reported incomplete/unsupported,
never silently omitted. Failed dataset reads preserve progress/reasons separately
from successfully captured comparable rows; no successful partial aggregate.

Lineage provenance (PW-14.07)

This dataset supplies no verified data-pipeline lineage. Return UNKNOWN lineage
coverage with an explicit UPSTREAM_LINEAGE_NOT_PROVIDED reason and no fabricated
edges. Sender/recipient relationships, forwarded text, matching subjects, duplicate
IDs and chronological proximity are not COPY/TRANSFORM or causation evidence.
Containment of train under the repository is C1 structure, not data lineage.
Existing diff/query/export must retain that uncertainty after offline reopening.

Evidence and initial review

All three actual downloads match their published SHA256/lengths. Parquet footers
each report172467 rows/173 row groups, totaling517401. Full selected-column audit
finds no duplicate file_name/message_id and no nulls in audited identity/date/list
parent columns; schema nullability is still respected rather than generalized away.
Recipient maxima include1029 to entries and457 cc/bcc entries. Exact dates/types
are in the audit; no record text is stored in Git for this decision.

Tests will use small synthetic Parquet fixtures with independent expected frames,
controlled changes and permission/read/version faults. Real-corpus outputs/cache
stay in ignored local artifacts; body text is not packaged as test data or printed
as routine diagnostics. These planning/data-audit changes do not claim a fresh
Python coverage pass. Q1 applies unchanged to every implementation subtask.

Primary sources consulted November2:
https://huggingface.co/datasets/corbt/enron-emails
https://huggingface.co/api/datasets/corbt/enron-emails
https://huggingface.co/api/datasets/corbt/enron-emails/tree/cfc06c758093d90993abce1a43668fb7357258a6/data?expand=true
https://arrow.apache.org/docs/python/generated/pyarrow.parquet.ParquetFile.html