An Interview: Jim Novo on the Web Analytics Association's Certification Program
Jim is the owner of The Drilling Down Project, his own analytics consultancy based out of Tampa, FL. He is the author of "Drilling Down, Turning Customer Data into Profits with a Spreadsheet," and co-author (with Bryan Eisenberg) of "The Marketer's Common Sense Guide to E-Metrics."Jim was instrumental in developing the Web Analytics Association's Certification program and currently sits on both the Certification Board and the Examination Board - About Certification.1. What is the difference between the Certification Board at the WAA and the Examination Board?The Certification Board handles the "rules and regs" part of Certification, the policy and ISO / ANSI standards compliance. The Exam Committee (not a Board, officially) focuses solely on developing questions.2. When I was at eMetrics in San Jose last May, the first web analysts were taking the new certification exam. How many have taken the test to date? How many have passed?12 have taken the exam and 10 have passed, there is a list of CWA's (Certified Web Analysts) here:http://www.webanalyticsassociation.org/?page=certifiedThe Test has only been available twice, at the eMetrics summits in San Jose and London this year, so access has been limited. We just launched the worldwide "test on demand" centers a couple of weeks ago so we expect those numbers to increase quite a bit over the next few years. Retaking the test is only $99 so we expect most who don't pass on first try will take the exam again. People can find out the locations of the 170 centers in the US and 60 global test centers here:http://www.webanalyticsassociation.org/?page=cert_apply3. Any themes that concern or surprise you so far in the feedback that you have received?Well, we're very pleased to see that top scorers have very different backgrounds and job descriptions - Technology, Marketing, Usability, Finance, Business Intelligence & Optimization. This is one sign of a balanced and fair test; you would not want success to be skewed towards one discipline or the other. Thankfully, no negative surprises, probably due to the fact we followed well-established best practices and standards for Certification Testing, including extensive testing of the test - thanks to all who helped with that!4. I am curious if more of the people that have taken the exam so far have been from the vendor side or the client side. There seems to be a quicker path to justifying the time, energy and expense on the vendor side (going to market with WAA Certified Consultants)…It's a varied mix of consultants, vendor-side and client-side folks. In the near future, we expect to see a lot more "big company" folks taking the test - vendors, agencies, clients - because free exam sessions are part of the corporate membership package, including discounts for bulk testing. Contact the Executive Director at WAA, Mike Levin, for more info on corporate membership packages:mlevin@webanalyticsassociation.org5. When I first read the criteria for each of the 3 areas of testing I was a little bit concerned that the test would actually be too hard. Within Marketing Campaign Management, for example, you are testing for:
From my experience, it is difficult to find professionals with expertise in any 2 or 3 of these, much less all of them. Can you explain how the test works? In other words, does an analytics professional have to be an expert in all these areas to be certified?Good point, and thanks for bringing this up.You don't have to be an expert in every facet of web analytics to pass the test, but it helps to have very broad (not necessarily always deep) knowledge of the entire web analytics ecosystem, including the business side.And we find this capability usually comes primarily from experience, not from studying books and taking courses. While it is possible a person with 5 years of experience has had a very narrow mission (say only PPC campaign analysis), we find most analysts with 5 years experience, and even some with only 3 years experience depending on their background, have been exposed to a broad range of analytical and business situations. If there is a practical area we find people to be weak in, it's using web analytics for Site Optimization (optimizing beyond the Landing Page), where most people are quite strong on Campaign Analysis. And many analysts could use some more experience with the cultural / project management issues surrounding web analytics.Here's another way to think of this issue. The test has two parts, short questions and case studies, both with multiple choice answers. The short questions section is a fundamentals kind of thing - do you know the difference between first and third party cookies, a visit and a visitor, etc. Most web analysts get pretty good scores on this section.The case study section is where many people get tripped up. The cases are based on real world situations experienced analysts have encountered in their day-to-day work, complex business scenarios where web analytics is used to make important decisions affecting the bottom line and / or business culture of the company. You can't "study" for these kinds of questions, they are designed to force you to prove you can do the job of an experienced web analyst, which is what Certification really means.You can see examples of the short questions and a case study from the actual test here:http://www.webanalyticsassociation.org/?page=cert_exam_res6. From a career-pathing perspective it is easy to understand the benefits of certification to the individual. But certification is also tremendously valuable to companies that are investing in analytics and want to drive ROI from those investments.We find that a lot of companies still don't know what they don't know about web analytics. There is nobody in the organization that has the expertise to determine a candidate's qualifications. The company has often made a significant investment in tools and technologies and they are looking for the talent that can bring the web analytics function to life.If the candidate walking through the door is certified (and certification is the standard of excellence in the community - not just competence), their task becomes much easier.Similarly, there are a lot of independent consultants, small consulting companies and interactive agencies that are trying to build web analytics groups. Presenting a team of WAA-certified consultants could be a major competitive advantage.What other groups are there that would benefit from certification taking root?I'd like to clarify a point of potential confusion before answering that question.Strictly speaking, if you follow Certification standards, a Certification is not a test of knowledge, it's a test of your ability to *do the job*. Many folks get confused by online testing that's offered as a "Certification" when really all it proves is you read some courses or material. That's not really a Certification, because it doesn't prove you can do the job. There's certainly nothing wrong with getting yourself educated and there's lots of good material out there, including the WAA's own courses offered online through the University of British Columbia.But this confusion about what a Certification really is has caused people to think they need to "study something" so they can regurgitate the content for the Certification Exam, and that's not true. Broad experience with web analytics is the best preparation you can have for the test. Second would be to working in an environment where web analytics is actualized in the business sense, and not just by distributing reports. Ideally you are creating new knowledge, providing insight, driving change, making a difference to the business with the output of your work. If that's what you are doing, and you are touching all of the web effort (not just email, or PPC, or Site Optimization, but all of it), you are ready to take the test.OK, to the question ;). The single largest area Certification can have an impact is in helping companies who have no internal experience with web analytics hire great analysts. This is the number one challenge we hear about, "mis-hires". Nobody in the company really knows how to interview a web analyst, so they have difficulty interpreting what they hear in an interview. To often the end result is "web analytics didn't work for us" when it's not web analytics that's the problem, it's the hire. It's in the best interest of the industry to reduce the number of "mis-hires".Another area with specific benefit is companies or government entities that must follow procurement guidelines. This is related to the idea above. Some companies just go ahead and hire even if they don't really have the experience to conduct a proper interview. Many large companies and especially government entities worldwide cannot do that. They need proper documentation or "proof" of a match between the job description and qualifications of a candidate. We've heard there's of a lot of demand out there that can't be satisfied because of this problem, and Certification raps that problem squarely on the head.7. What is the WAA doing to "sell" certification to its constituents and members? What are your goals in terms of the number of certified analysts that you would like to have?Fortunately most web analysts live online and I'm pretty sure there's wide awareness of the Certification Exam among practitioners, e.g. those who know what the tag "#measure" means. But I also think these same people may not really understand the true purpose of the Exam and what's required to pass it (as explained above), so outreach efforts like will continue and help a lot (Thanks!).From a broader industry perspective, we're pretty sure businesses in general are not aware of the Certification, particularly those that are not 100% online, which is a ton of them. So the WAA is planning and already engaging in outreach efforts through trade associations, governmental entities, and through direct business contact to spread the word. If you're a member of a trade association or government entity who thinks their members might benefit from a better understanding of web analytics in general and the Certified Web Analyst (CWA) credential, please contact the Executive Director at WAA, Mike Levin:mlevin@webanalyticsassociation.org
