Developer Surveys and Research

Developer surveys and research are the recurring public studies, such as Stack Overflow's and JetBrains' annual surveys and GitHub's Octoverse, that measure what software developers use, prefer, and struggle with, and the skill of reading their methodology critically before trusting a headline number.

itTechnical communication and collaboration

Don't Panic: Developer Surveys and Research

Every year, several large organizations ask developers what they use, and then several other organizations quietly disagree with each other about the answer. This is not a scandal. It is the whole subject.

Here is the thing nobody tells you up front: none of these studies are trying to find the same fact. Stack Overflow and JetBrains run self-selected surveys, meaning they post an open link and count whoever shows up, mostly their own already-engaged users. GitHub's Octoverse does not ask anyone anything; it just totals up commits and repositories that already happened. SlashData recruits deliberately across many channels and does the statistical equivalent of evening out the numbers afterward, which the field calls weighting. The TIOBE index does not go near a developer at all. It counts how often people type "Python programming" into a search box, which measures curiosity, not employment.

Four instruments, one population, four different questions. That is why a headline claiming "80 percent of developers use X" should make you ask, gently but firmly, "according to whom, doing what?"

The one thing that will surprise you is how honest the culprits are about this. JetBrains states plainly, in its own methodology page, that JetBrains users were probably more willing to fill out a JetBrains survey. Stack Overflow says the same about itself. Nobody is hiding the skew. The skew is footnoted, in public, every year, and most of the internet quotes the headline number one paragraph above the footnote anyway.

The surprising part is not that the numbers can be biased. It's that the bias is disclosed and gets ignored regardless, all the way from a methodology page through a blog post through a conference slide, each retelling a little more confident and a little less accurate than the last.

You do not need a statistics degree to fix this. You need one habit: when you repeat a number, say where it came from. "In Stack Overflow's self-selected 2025 survey" is six words longer than "developers say," and it is the difference between a claim someone can check and a claim someone has to trust.

Two tabs answer two different halves of this. The Cheatsheet lists which study uses which method, so you can match a claim to its instrument in ten seconds. The Field Notes cover the sharper judgment calls: which mistakes are common, and which shortcuts cost you more than they save. Between the two, you will never again have to take a headline's word for what developers do.

Don't Panic: Developer Surveys and Research

Relevant careers