Google AdWords - Change History, Logs Processing

Change History

Change History is a feature of AdWords where advertisers can view all changes made to their ad campaigns (changed via API and manually). I made full-stack changes to migrate all Ads change history data to a new database and serve this new data to user.

Accomplishments

• Wrote mapreduce pipeline for pre-rendering all historical change data and storing into new database. This allows deprecation of old database.
• Wrote mapreduce pipeline for verifying all migrated data is present for confidence before turning down the old database.
• Changes to frontend for serving from new database and storage format.
• Assisted with design decision of how to store migrated dta.

EvenFlow

EvenFlow is the Google Ads real-time processing pipeline that converts raw event logs for views and clicks into stats and metrics used for analysis and billing. I worked on the integration test suite for the end-to-end logs processing pipeline.

Accomplishments

• Improved pipeline instrumentation to record duration of each pipeline stage for each run.
• Created tool to generate graph showing runtime broken down by pipeline stage, showing parallel and serial sections.
• Created visualization to show total pipeline latency changes over the last few months. Lead to identifying increased latency over time.
• Improved regression test speed by running more stages in parallel, cutting down overhead time for a section.
• Reduced regression test resource utilization by creating system to store golden run data instead of recomputing each run.