This spring I joined the growing list of WAA Certified Web Analysts ™. While we’ve understandably been sworn to secrecy not to share the contents of the exam itself, some details about the experience might be useful for anyone considering taking the exam. I’ve heard several comments similar to ‘why bother’ or ‘is it really worth it?’ While it’s still too soon to weigh in on the latter, the simple answer regarding the former is that there is an extremely broad range of skill and knowledge in the web analytics field. Particularly as an independent consultant, I appreciate the opportunity to prove my depth and breadth of experience. I am thankful for the many hours the Web Analytics Association, along with scores of volunteers in the field, invested to launch this program. I understand that it’s extremely difficult to design a great test. While it’s a solid start, it’s not a great test.
I opted to take the exam at a remote testing center run by AMP (Applied Measurement Professionals). When I logged into the scheduling calendar, I was surprised but relieved to find out that the nearest testing center (just seven miles from my house) could accommodate me any day of the week, morning or afternoon, for the foreseeable future. The abundance of choice lulled me into complacency, and it took several months to force myself to log back in to lock in a date.
The morning of my scheduled exam, I pulled into the parking lot of the nondescript H&R Block office whose second identity turns out to be an AMP testing center. The dying local shopping center (photo below) across the street from a remote testing center certifying a profession that did not even exist when I went to college provided a stark convergence of the new and old economy.
A professional and friendly receptionist greeted me and another test-taker at the door. I gathered he was taking a different exam since he was permitted a calculator. We were instructed to bring our car keys, his calculator, and nothing else (they even provided pencils), into a back room with four computer workstations in cubicles.
The 90-minute exam consists of a series of multiple choice questions, followed by additional multiple choice questions about several case studies. The testing software is very straight–forward, and permits marking questions for review. I initially thought this would be helpful, until I realized I was marking at least a third of the questions. Many of the questions have two answers that can be easily eliminated and then two where I struggled to figure out the intent of the question to help narrow down the answer. I wished I could write an essay response about why I was stuck between the two remaining answers, and the reasons why I might pick each one. But of course, in a multiple choice exam there is only one right answer, and no partial credit for getting close. Several questions seemed unnecessarily picky. For example, one listed several important tasks to do when starting a new job, and asks which should be done first. If two answers are both things to do during the first week, does it really matter which one you do ‘first’? I think a solid web analyst could still fail this exam.
While those who pass definitely demonstrate a certain level of competency, is the exam able to test the qualities that make a great web analyst? Absolutely not. It can’t figure out who can communicate well to developers, the marketing team, the product team as well as all levels of executives. It can’t figure out who has the patience to run QA on web analytics tags, page by page, click by click, to diagnose a data collection issue. It can’t tell you who really enjoys peeling the onion to find the underlying segment or source that’s driving a change in a KPI. It doesn’t tell you who truly understands the business needs as well as the technical challenges to collect the data.
The exam does nevertheless screen for a basic body of knowledge and general analytic capability. It provides an indication of experience and quality for employers, and for companies considering hiring a web analytics consultant. For that, I am grateful.