The Start

Scaling a mindset of experimentation at IBM Marketing has required a significant cultural shift that gives people permission to fail fast with a base knowledge of the scientific method and hypothesis building.

What was a centralized program in 2015 owned by one person – resulting in an undemocratic process for a practice meant to be collaborative – transformed into one that scaled across 6,000 marketers worldwide thanks to the Testing & Personalization (T&P) program in Performance Marketing.

Leveraging Optimizely, our 2016 and 2017 strategy focused on the velocity and volume of experiments as we kicked-off global contests for the most novel ideas, created a custom dashboard to guide marketers on where to test and who was leading the charge in the leaderboard, held regularly scheduled training programs, and created a consistent cadence of support as the program's foundation.  IBM's eminence in the world of testing and personalization grew as we were invited to speak at conferences like Opticon to share our story. 

The Scale

In 2018, we introduced the power of personalization to IBM Marketing, with our first ever personalization pilot being targeted to the developer audience on the US Homepage, garnering 1700% uplift in our primary KPI.

The frenzy for adoption gave us the momentum to implement GDPR-compliant personalization at scale with this proof point. To support the mass adoption, we created a Center of Excellence (CoE) to consult with marketing teams on the end-to-end process from analysis to experiment execution.

To further instill the virtue of the scientific method, former IBM CMO Michelle Peluso kicked off the Quarter of Client-Centricity in Q2 2018. The number of experiments that year increased 4x the number in 2017. 2019 held fast to the momentum and doubled the number of tests and personalization activities from 2018. 

The Shift

If 2016-2019 were about quantity, our 2020 focus was on experiment quality and impact.

The T&P program was involved in some of the biggest 2020 initiatives to address IBM's response to COVID-19 and market it in a data-driven way.

A combination of customer insights on awareness and receptivity to the Emerge Smarter initiatives along with rapid experimentation on the digital experience helped evolve our COVID response strategy.

On the program level, we also began to experiment with a framework that would support a systematic approach to experimentation coordinated across the Business Units, in addition to the steady-state ad-hoc experiments, to identify scalable best practices for web across BUs. This systematic approach is one we're continuing to champion in 2021 with the Adobe transformation in Project Afterburner.