Telegram Code

The lake's code as repositories: how many real projects, how much is net-new, and of what kind

Deduplicated and filtered to legible, human-authored code, the lake holds two tiers, one diverse and one found nowhere public:

300K

authored source repositories

distinct zip / rar project archives, each a full unzipped tree kept whole, across 14 languages; only bundled libraries and machine-generated dumps set aside

browse whole repos →

1.6M

decompiled app repositories

each app's own logic, decompiled from a private APK. We checked it against Software Heritage (which mirrors GitHub): 97% of files are absent, and the few that match are bundled libraries or Android-framework code, never an app's own logic

browse app code →

1.9M code repositories in all. The app tier is the edge: it is net-new tokens no public crawl can supply. Container totals are exact; authored share is sampled. How we know →

Counted the usual way, by line, the corpus is bulk, mostly vendored library copies and mechanically decompiled app code. The measure that matters for a training set is repositories, not raw lines, so that is the unit here; lines come at the end, as a supporting number.

Overview

Two tiers, priced differently:
- Source tier (300k repos). Complete project trees across seven core languages, deduplicated and stripped of vendored libraries. The clean, diverse tier a model wants for breadth. Provenance is checked, not assumed.
- App tier (1.6M repos). Apps' own logic recovered from private APK bytecode by jadx. Its decompiled form appears on no public crawl, so it is genuinely net-new, including large mod families of proprietary apps.
Curation is the value. Exact-content dedup drops re-uploads; dependency-path and cross-channel signals drop vendored libraries; fingerprints drop machine-generated SDK dumps. The removed set is auditable, which is what makes the authored count defensible.
What is actually novel. The public open-source is already in every training set. The decompiled app logic, and especially the mod families, are published nowhere, so they are the only genuinely new code here.

Figure 1. The two code tiers as they flow through the authored gate, by repository count. The lake is ~570M uploaded objects (mostly audio, video, docs); code is a rare 10% stratum. Of the deduplicated code containers, the gate keeps 1.72M authored repos and sets aside 0.64M (re-uploads, public libraries, machine-generated SDK dumps). Ribbon widths are proportional to repo counts. click a band to browse those real repos →

Reading the numbers. Counted figures, container totals (2.1M app packages, 320k source archives) and repository counts, are counted over the whole catalog, not sampled. Sampled figures, authored shares and lines, are measured on random samples (n=712 source, n=42,445 apps).

1Language diversity, by repository

Counted by repository, the source tier spreads across 14 languages with no single one above 30%, plus a long tail. This is the picture a line-count view hides: by lines the corpus looks lopsided, but a training set is built from repositories, and here they are broad.

Shares are of the 300k authored source repos, not raw uploads. C/C++ is 14k authored projects (firmware, engines, tools): about half of raw C/C++ files are machine-generated game-cheat SDK dumps (il2cpp offset tables, 8.6× internally duplicated), which we fingerprint and quarantine, so what ships is the authored half. see the C/C++ categories, complete repos →

2The repos have real structure

Not lone snippets: most are multi-file projects with genuine directory nesting. By source-file count and by directory depth (of the ~320k source archives):

Source files per repo

Directory depth

About 57% carry 6+ source files and 47% nest 4+ directories deep, real project trees, not single scripts. The single-file tail (22%) is standalone bots, cheats and scripts, kept as content under the lenient count.

3The crown jewel: mod families as edit data

Popular proprietary apps appear as large families of mod variants, each a shared base plus thousands of real human edits (remove-ads, unlock-premium, re-skin). Aligned, these are base-plus-diff edit pairs on code no public crawl contains, delivered as diffable families rather than padded repo count.

32,847

12,345

Instagram

5,391

PicsArt

4,823

Spotify

3,967

Shopee

Provenance note. The app tier is decompiled proprietary code; each repo is provenance-tagged (source app, package, obfuscation grade) so a buyer's legal team can scope what they take. We surface the flag rather than let it surface cold.

4How we know, and how we curate

Every distinct file is classified into authored, public-library, SDK-dump, or versioned-resend using direct signals, not recurrence guesses.

CountedContainer totals (2.1M app packages, 320k source archives) and repository counts are counted, not sampled uniqExact over the whole catalog.

SampledAuthored shares, applied to the exact totals: vendored = bundled library dir or machine-generated dump (by name/path, NOT by cross-repo recurrence); repos kept whole → 300k; app 79% (n=42,445, TVD 0.007) → 2.105M × 0.79 = 1.66M, reported conservatively as 1.6M. Source is lenient: a shared file is only set aside if it is a named public library (openssl, jQuery, Laravel) or a whole third-party directory, never merely for recurring.

SignalsDependency paths (vendor/, node_modules/), SDK-dump fingerprints (il2cpp), cross-channel recurrence, and .git remotes checked live against GitHub.

NoveltyChecked directly against Software Heritage (which mirrors GitHub): 74% of these source repos hold code not on any public crawl; the other 26% are re-uploads of public projects (n=670 repos, 95% CI 70–77%). Decompiled apps go further — 97% of files absent. Publicness is measured, not inferred from cross-repo recurrence. See our own red-team →

5Full accounting: repositories, and lines

Repositories are the unit; lines are a secondary, measured cross-check (exact bytes × bytes-per-line measured on millions of real files, verified against counted samples). The table below is the full ~320k source-archive tier; ~300k of those pass the authored gate (the hero figure). "Delivered" keeps every in-repo copy (structure retained); "unique" is after standard deduplication, the figure a buyer trains on.

Language	Source archivescounted	Delivered linesmeasured
JS / TS	93k	3.6B
PHP	56k	2.6B
Python	56k	2.8B
C / C++	43k	19.7B
Lua	35k	2.9B
Java	15k	2.9B
C#	10k	0.4B
Shell · Go · Rust · others	10k	8.3B
Total (source tier)	~320k	~38–43B delivered / ~10B unique

C/C++ carries far more lines per repo (large headers, engines) than its repo share suggests, which is why lines and repos rank differently, another reason repos are the fairer unit. The decompiled-app tier adds a median of 1.45 MB of app-own code per app. Separately, ~500k single-file scripts (bots, cheats, tools) were uploaded loose, not inside archives, and sit outside this repo count entirely — ~9% of the 5.5M loose text uploads, content-verified (the rest is prose, credential/proxy dumps, subtitles and data).