Work · Analysis · 01
German Job Market Intelligence Corpus
What 3,200 German tech job postings actually ask for, measured rather than guessed.
- Type
- Analysis
- When
- FOM M.Sc. · 2026
- Role
- Sole author. Scraping, cleaning, gazetteer construction, extraction, clustering, write-up.
- Scale
- 4,183 raw postings → 3,200 validated · 156 skills extracted
In plain language
Careers advice for the German tech market runs on anecdote: learn Python, get good at SQL. Reasonable, but nobody had checked. Job adverts are the one place employers state in writing, at scale, what they will actually pay for, so I collected 3,200 validated German tech postings and read all of them with software. The headline is that Python and SQL appear together in 65% of postings: not two useful skills, but an empirical minimum pair.
The question
What do German tech employers actually ask for, measured from their own job adverts rather than inferred from discourse?
Job adverts are the only place where employers state, in writing and at scale, what they will pay for. Nobody reads 3,200 of them. So I built something that did.
The problem
Careers advice for the German tech market runs on anecdote. “Learn Python.” “Get good at SQL.” Reasonable, but unmeasured, and when you are an international student deciding what to spend a semester on, the difference between advice and evidence is the difference between a job and a wasted term.
Approach
4,183 raw postings scraped from StepStone, Indeed DE and LinkedIn, deduplicated and language-filtered down to 3,200 validated records. Skill extraction ran a spaCy EntityRuler over a hand-built gazetteer of 156 tools and techniques, validated at 88.4% precision against a manually labelled sample. TF-IDF vectors then fed K-Means, with k chosen at 4 on a silhouette score of 0.61.
What it found
Python and SQL co-occur in 65% of all postings. Not “both are useful”. They are the empirical minimum viable pair. A posting that wants one almost always wants the other, and the roles that want neither are a different job family entirely.
The four clusters split cleanly into analytics, engineering, science and platform roles, and the cloud requirement is where they diverge most sharply: Azure dominates the German market in a way it does not dominate the English-language discourse about it.
What I took away
The clustering mattered less than the co-occurrence matrix. A silhouette of 0.61 is respectable but not decisive, and I would not stake a claim on the cluster boundaries. The pairwise co-occurrence numbers are simple, robust, and directly actionable, which is the more useful result even though it is the less sophisticated one.
How it is put together
Left to right, the path a request or a record takes through the system.
01
Collect
- StepStone
- Indeed DE
- BeautifulSoup
Three boards, because one board is one board's bias.
02
Clean
- Deduplication
- Language filtering
- 3,200 validated records
4,183 in, 3,200 out. The gap is the work.
03
Extract
- spaCy EntityRuler
- Hand-built gazetteer, 156 tools
- 88.4% precision on a labelled sample
04
Analyse
- Co-occurrence matrix
- TF-IDF vectors
- K-Means, k=4, silhouette 0.61
What happens, step by step
In plain language, in the order it happens.
Scrape three job boards
4,183 raw postings from StepStone, Indeed DE and LinkedIn. Three sources rather than one, because each board attracts a different slice of employer.
Cut it down to what is real
Deduplication and language filtering take it to 3,200 validated records. Reposted and cross-posted adverts would otherwise weight a single employer's preferences several times over.
Extract skills by rule, not by guess
A spaCy EntityRuler over a hand-built gazetteer of 156 tools and techniques, validated at 88.4% precision against a manually labelled sample.
Count what appears with what
The pairwise co-occurrence matrix is the simplest analysis here and turned out to be the most useful one.
Cluster the roles
TF-IDF vectors into K-Means, k chosen at 4 on a silhouette score of 0.61: analytics, engineering, science and platform.
Decisions, and what they cost
Every choice worth recording has a road not taken. Both are here.
A hand-built gazetteer with rule-based extraction
instead ofTraining a NER model on the corpus
With 156 well-defined tool names, rules give higher precision than a model trained on 3,200 documents, and every extraction is auditable back to a rule. A trained model would have been more impressive to describe and less trustworthy to use.
Report the co-occurrence matrix as the headline
instead ofLeading with the clustering
A silhouette of 0.61 is respectable but not decisive, and I would not stake a claim on where the cluster boundaries sit. The pairwise co-occurrence numbers are simple, robust and directly actionable, and the more useful result even though it is the less sophisticated one.
Three job boards
instead ofOne clean source
Each board over-represents a different employer type. Merging three and deduplicating costs a cleaning stage and buys a defensible claim about the market rather than about one platform.
Validate precision against a manual sample
instead ofReporting extraction counts unchecked
An extractor that quietly misses a third of mentions produces a confident and wrong ranking. The 88.4% figure is what makes every number downstream of it worth quoting.
What came out of it
- Python and SQL co-occur in 65% of all postings: the empirical minimum viable pair, not a matter of taste.
- 156 skills extracted at 88.4% precision across 3,200 validated postings.
- Four role clusters: analytics, engineering, science, platform, separating most sharply on cloud requirement.
- Azure dominates the German market in a way it does not dominate the English-language discourse about it.
Still open
- Postings state what employers ask for, not what they hire on. The gap between the two is unmeasured here.
- One snapshot in time. The same pipeline run quarterly would say something about direction, which is the more useful question.
Built with
- Python
- spaCy
- scikit-learn
- TF-IDF
- K-Means
- BeautifulSoup