How much code is in the media lake, in what languages, and of what kind
Abstract
The lake holds 482B lines of code across 4.7B files in 12+ languages.
Counts preserve repository structure: exact re-uploaded archives are removed, but every file inside a kept project is counted (the
archive-deduped tier). The totals are measured from real files, not estimated (see How we know these numbers).
Code comes in three kinds:
vendored third-party libraries copied into projects (54%), Android app logic decompiled from bytecode
(44%), and original human-written source uploaded as-is (only 2%).
Overview
The three kinds of code:
Vendored (258B, 54%). Third-party libraries copied into projects. In decompiled apps, these are bundled SDKs; in
archive source, the same files copied across thousands of projects (openssl appears in 6,394, imgui in 4,129, plus game and Unity
SDKs). Measured by library package name and by cross-repo recurrence.
Decompiled app code (213B, 44%). Apps' own logic recovered from APK bytecode by jadx. Readable, structured
code; about a third has names stripped by an obfuscator, though the logic stays intact.
Raw authored source (10B, 2%). Original human-written source unique to one project. The smallest slice.
Dedup keeps repo structure. We drop exact re-uploaded archives but keep a library copied into many projects, since it is
part of each project's tree. Tiers are in Table 1.
What is actually novel. The vendored 54% is public open-source (openssl, imgui, boost), already on GitHub and in training
sets. The decompiled app logic (44%), reconstructed from private APK bytecode, is published nowhere, so it is the part not otherwise
available. Only that plus the ~1% genuinely-original uploaded source is new.
Figure 1. How the 482B distributes, from how it arrived (decompiled from APKs, or extracted
from archives) to what kind of code it is. More than half is vendored library copies; only a sliver is original authored source.
vendored / public re-upload (removed from "authored")
Each cell opens the code viewer showing files of that kind.
Figure 2. The two corpora split by origin, with clickable cells. Only ~48% of the decompiled bar and
~26% of the raw bar is plausibly uploader-authored; the rest is bundled SDKs, shared libraries, and copied libraries.
Every cell is a link to the browsable files.
1By language
Java dominates because it is decompiled from nearly every Android app. The rest comes from uploaded source archives and is
diverse: PHP has the most files (30M small web files), Shell has few files but long build scripts. Figure 3 shows the
archive-source languages by lines; Table 1 gives the exact counts.
Figure 3. Archive-extracted source by language, area proportional to lines (decompiled Java is
shown apart, it would dwarf the map). Hover for files and share.
Table 1. Per language: files, lines, and the share that is
vendored (the same file copied across many projects). Java's vendored share is bundled SDKs; C/C++ is copied headers.
Table 2. Repos by primary language across all archive types (zip, rar, other), each distinct repo assigned to its single dominant language, no overlap. Shell is set aside here: it is ~214k tiny module and installer packages (Magisk, OpenWrt, cheat installers) that dominate the repo count but are only 1.5% of code by line (Table 1). The ~252k substantive code repos are shown; the ~1.9M decompiled APKs are a separate corpus.
Primary language
Repos
Share
JS / TS
71,986
29%
Python
44,103
17%
PHP
43,141
17%
C/C++
39,055
15%
Lua
27,602
11%
Java
13,557
5%
C#
8,800
3%
Go / Rust · Ruby · SQL
4,052
2%
Substantive code repos
252,296
Loose code files (a single script posted directly, not inside an archive) are excluded: once binaries mislabeled with code extensions are removed (for example .ts video), genuine loose code is under 1% of lines.
2How we count
Counts collapse genuine duplication but keep repo structure. Three tiers:
Gross: every occurrence.
Content-distinct: every identical file collapsed once (the unique-code floor).
Archive-deduped (reported): drop exact re-uploaded archives, keep every file inside a kept project.
The gap between tiers is vendoring. Java runs 148B distinct to 442B kept (androidx/kotlin bundled into
each APK); C/C++ runs 4.3B distinct to 17.7B kept (headers copied into 7 to 8 times as many projects as unique copies).
3How we know these numbers
The corpus is billions of files, far too many to read in full, so we sample and measure, then check the sample is large enough
that more data would not change the answer. Four things make the totals trustworthy:
Unbiased sample Files are drawn by their content fingerprint, with no size cap, so nothing is favored: not large files, not duplicates. Each distinct file is an independent pick.
Counted, not guessed We fetched real files (2,041 Java, 540 archive) and counted their lines directly. Every fetched file's size matched the catalog exactly, so the byte totals are trustworthy and the line totals are exact, not estimated.
Stable in every cut The breakdown, and counts like the 627k archive total, are computed directly over the whole catalog (an exact uniqExact), never scaled up from a sample, so a naive ×N projection cannot over-count. The shares are exact. As a stress test we split the data in half and compare all 33 tags at once (language, origin, obfuscation): the halves agree to a total-variation distance of 0.05, and that gap halves each time we quadruple the data, so no single cut is undersampled.
Right buckets The authored-vs-vendored split is confirmed by a signal it was not built from: authored code sits in one project, while library code recurs across thousands (openssl in 6,394).
Sampled per corpus The two corpora likely have different distributions with different variance, so one sampling rate is wrong for both. Decompiled Java is huge but low-variance (a few buckets), so a light sample pins it; archive source is small but high-variance (dozens of language-by-origin cells), so we sample it near-fully. Stratifying keeps every cell adequately sampled instead of over-sampling Java and starving archive.
Every number is measured from the lake catalog and files fetched through the bastion, at the archive-deduped tier.