Testing, 1-2-3, Testing
During the past several months I’ve had an interesting experience working with Brainbench. As you may know, Brainbench (a part of Previsor) offers assessment tests and certifications across a wide range of subjects. They cover many technical and non-technical areas. I have taken Brainbench exams myself and I have seen them used as a component within a hiring process. However, I did not understand how these exams were created.
That mystery ended for me late last year when I received an email looking for technologists to assist in validating a new exam that Brainbench was creating to cover Spring version 2.5. Being curious about the test creation process I applied for the advertised validator role. I was pleasantly surprised when they contacted me with an offer for the role of test author instead.
I will not delve into Brainbench’s specific exam creation approach since I assume it is proprietary and want to be sure I respect their intellectual property. What I found was a very well-planned and thorough process. Having a background in education and a strong interest in teaching and mentoring, I know the challenge of creating a meaningful assessment. In the case of their approach, they focus on an accurate and well-considered exam.
I believe that I am quite knowledgeable regarding Spring. I have used many of its features for work and personal projects. The philosophies supported by the product (encouraged, not prescribed) address many areas of coding that help reduce clutter, decouple implementations, and simplify testing. As a true fan of Spring’s feature set, I found it challenging to decide which aspects were most important when assessing an individual’s knowledge of the overall framework.
What was great about the Brainbench process was that I was given a structure with which to document my thoughts about how to create the test. My opinions were then given to a team of validators (other Spring experts) with whom I could collaborate through Brainbench’s test design environment.
That collaboration process worked well as the test took form. Beyond determining the different parts of Spring to be assessed, the collaborative environment allowed the validators and me to verify code sample intelligibility, question clarity and answer accuracy.
Although I have used Spring for years to deliver a variety of solutions, it was an educational experience to put myself in the role of an instructor and consider how I would verify an individual’s knowledge of its features. Probably the greatest challenge for me was taking the topic I wanted to assess and creating an effective way to measure someone’s knowledge of that topic.
Creating the measure involved setting up a very precise code example, wording a concise question and providing an accurate answer. Beyond those components, and for me the hardest step, I had to define a set of additional answers that seem reasonable but were not correct.
This aspect of the test creation, providing incorrect answers, is necessary since the exams are multiple choice. The Brainbench editors provide a great deal of assistance to authors so that the correct answer doesn’t end up being obvious. I knew my subject, but creating “good“ incorrect answers was hard work! I was thankful to have the suggestions of the validators available to me as we worked through a large set of questions.
Having completed the process, I have a lot of confidence that the test is well constructed. Of course the true verification will occur when developer’s actually take the test. Based on my experiences with Brainbench, I know that tests go through a Beta period. I’ll be curious how this test is received.
Overall the experience was fun and quite educational. The process was personally rewarding and I hope that companies and individuals benefit from the exam when it becomes available.
Relevant sources:
- Spring: http://springsource.org/
- Brainbench: http://brainbench.com/
Tags: assessment, Brainbench, education, skill testing, Spring