Logo for adwords

Google AdWords - Change History migration, logs processing integration testing

Change History

Change History is a feature of AdWords where advertisers can view all changes made to their ad campaigns (changed via API and manually). I made full-stack changes to migrate all Ads change history data to a new database and serve this new data to user.

Accomplishments

  • Wrote mapreduce pipeline for pre-rendering all historical change data and storing into new database. This allows deprecation of old database.
  • Wrote mapreduce pipeline for verifying all migrated data is present for confidence before turning down the old database.
  • Changes to frontend for serving from new database and storage format.
  • Assisted with design decision of how to store migrated dta.

EvenFlow

EvenFlow is the Google Ads real-time processing pipeline that converts raw event logs for views and clicks into stats and metrics used for analysis and billing. I worked on the integration test suite for the end-to-end logs processing pipeline.

Accomplishments

  • Improved pipeline instrumentation to record duration of each pipeline stage for each run.
  • Created tool to generate graph showing runtime broken down by pipeline stage, showing parallel and serial sections.
  • Created visualization to show total pipeline latency changes over the last few months. Lead to identifying increased latency over time.
  • Improved regression test speed by running more stages in parallel, cutting down overhead time for a section.
  • Reduced regression test resource utilization by creating system to store golden run data instead of recomputing each run.
Visit Google AdWords