Engineering Online Experimentation and ML Evaluations
Category: eBooks
- views: 50
- date: 30 July 2026
- posted by: BeMyLove
Engineering Online Experimentation and ML Evaluations | 19.79 MB
Title: Engineering Online Experimentation and ML Evaluations
Author: Ming Lei
Language: English | 527 Pages | ISBN: 979-8868827204
Description:
Online experimentation is now essential for modern software and machine learning teams. This book provides an engineer-first, end-to-end guide to building and operating production-ready experimentation platforms.
The book begins with Part I establishing the core foundations of credible experimentation, including hypothesis testing, power analysis, sample sizing, metric design, and common pitfalls such as peeking, multiple testing, and novelty or learning effects. Part II focuses on platform engineering—traffic and identity management, mutual exclusion, event and logging design, ETL/ELT pipelines, building a stats engine with SciPy and statsmodels, SRM detection, integrating deployments with feature flags and canaries, and setting up guardrail and health monitoring. Part III presents advanced designs that improve speed and sensitivity: sequential testing with alpha spending, bootstrap intervals for ratios and quantiles, A/B/n testing with ANOVA, interleaving for ranking systems, switchback and geo experiments, and multi-armed bandits. Part IV connects experimentation to ML workflows, covering offline, shadow, canary, and A/B evaluation pipelines; Bayesian optimization for adaptive experimentation; counterfactual and IPS methods for learning from logs; and safe retraining supported by strong governance.
What you will learn:
Who this book is for:
The primary audience for this book includes Data Engineers, ML Engineers, and Platform or Software Architects. It is also well suited for Product and Data Scientists who want a deeper understanding of experimentation systems and the engineering principles behind them.
DOWNLOAD:
https://rapidgator.net/file/a73f5990dd037b1292aff26b3c571aa7/979-8-8688-2721-1.rar
https://nitroflare.com/view/85A14B6D0E25F2B/979-8-8688-2721-1.rar
Online experimentation is now essential for modern software and machine learning teams. This book provides an engineer-first, end-to-end guide to building and operating production-ready experimentation platforms.
The book begins with Part I establishing the core foundations of credible experimentation, including hypothesis testing, power analysis, sample sizing, metric design, and common pitfalls such as peeking, multiple testing, and novelty or learning effects. Part II focuses on platform engineering—traffic and identity management, mutual exclusion, event and logging design, ETL/ELT pipelines, building a stats engine with SciPy and statsmodels, SRM detection, integrating deployments with feature flags and canaries, and setting up guardrail and health monitoring. Part III presents advanced designs that improve speed and sensitivity: sequential testing with alpha spending, bootstrap intervals for ratios and quantiles, A/B/n testing with ANOVA, interleaving for ranking systems, switchback and geo experiments, and multi-armed bandits. Part IV connects experimentation to ML workflows, covering offline, shadow, canary, and A/B evaluation pipelines; Bayesian optimization for adaptive experimentation; counterfactual and IPS methods for learning from logs; and safe retraining supported by strong governance.
What you will learn:
- Design trustworthy experiments with proper metrics, guardrails, α/power/MDE settings, and safeguards against peeking and multiple-testing errors
- Build a production-ready experimentation stack with assignment, identity/diversion, logging, ETL/ELT, a stats engine, and SRM checks
- Run advanced designs at scale, including sequential tests, bootstrap CIs, interleaving, switchback/geo experiments, and multi-armed bandits
- Evaluate ML systems from offline to online, leverage experiment logs for learning, and enable safe retraining with governance
Who this book is for:
The primary audience for this book includes Data Engineers, ML Engineers, and Platform or Software Architects. It is also well suited for Product and Data Scientists who want a deeper understanding of experimentation systems and the engineering principles behind them.
DOWNLOAD:
https://rapidgator.net/file/a73f5990dd037b1292aff26b3c571aa7/979-8-8688-2721-1.rar
https://nitroflare.com/view/85A14B6D0E25F2B/979-8-8688-2721-1.rar
We need your support!
Make a donation to help us stay online
Bitcoin (BTC)
bc1q08g9d22cxkawsjlf8etuek2pc9n2a3hs4cdrld
Bitcoin Cash (BCH)
qqvwexzhvgauxq2apgc4j0ewvcak6hh6lsnzmvtkem
Ethereum (ETH)
0xb55513D2c91A6e3c497621644ec99e206CDaf239
Litecoin (LTC)
ltc1qt6g2trfv9tjs4qj68sqc4uf0ukvc9jpnsyt59u
USDT (ERC20)
0xb55513D2c91A6e3c497621644ec99e206CDaf239
USDT (TRC20)
TYdPNrz7v1P9riWBWZ317oBgJueheGjATm
Information |
|||
 |
Users of GUESTS are not allowed to comment this publication. | ||