Feature Flags are software switches that turn on or off a feature - usually in real-time, without needing to release a new version of the software.
Building great products is not easy. Getting product market fit is even harder. This usually requires multiple iterations based on product feedback to get it right.
Calculating the required sample size for an A/B Test (also known as a split test or bucket test) helps you run a properly powered experiment.
Canary release is a technique of slowly rolling out the change to a small subset of users and testing it before rolling it out to everyone.
A comprehensive guide to getting started with your first feature based on Feature Flags/Gates/Toggles.
Also known as split or bucket testing, A/B testing is the scientific gold standard for understanding and measuring causality
Uber’s many software systems require a high volume of changes every day. Because of our systems’ size and complexity, it is a significant challenge to implement these changes without unintended consequences, ultimately slowing down developer productivity.
At Spotify we try to be as scientific as possible about how we build our products. Teams generate hypotheses that we test by running experiments — normally in the form of an A/B test — to learn what works and what doesn’t. The learnings give us insights and fuel new product ideas.
https://engineering.atspotify.com/2020/10/29/spotifys-new-experimentation-platform-part-1/
Over time, the software industry has come up with several ways to deliver code faster, safer, and with better quality. Many of these efforts center on ideas such as continuous integration, continuous delivery, agile development, DevOps, and test-driven development.
https://engineering.fb.com/2017/08/31/web/rapid-release-at-massive-scale/
While the basic principles behind controlled experiments are relatively straightforward, using experiments in a complex online ecosystem like Airbnb during fast-paced product development can lead to a number of common pitfalls. Some, like stopping an experiment too soon, are relevant to most experiments.
https://medium.com/airbnb-engineering/experiments-at-airbnb-e2db3abf39e7
Your product’s metrics are crashing; revenue is down 5% week-over-week, and daily active users are down 4%. You know Team A shipped a new product ranking algorithm, Team B optimized the payments flow, while the marketing team overhauled their retention campaign. Meanwhile your analysts remind you that it’s the start of summer and to expect “some” seasonality. How will you determine what’s TRULY driving the crash?
https://blog.statsig.com/why-a-b-testing-is-so-powerful-for-product-development-eca7c050c40a
