As volunteer examiners we see people getting their ham radio license by memorizing the answers using practice exams, learning little in the process, then dropping out of ham radio because they don’t know where to begin.  If you’re going to invest all those hours studying for the exam, most people want to learn something about ham radio in the process.

Practice exams do serve a useful purpose, proving your readiness for the exam after you have completed a course of study.  However, they should not replace education.

This article explores some of the issues with getting a ham license using only practice exams.  We begin with an examination of how the actual exams are structured.

The National Conference of Volunteer Examiner Coordinators (NCVEC) maintains the question pools used for all U.S. amateur radio examinations.  The FCC regulations require at least ten times as many questions in the pool as appear on the actual exam, but the NCVEC generally exceeds that requirement by a considerable margin.  As required by the FCC, the NCVEC publishes the question pools to the general public well in advance of their effective date.

Each U.S. question pool is divided into 10 topics, each topic into between 1 and 8 groups, and each group contains between 10 and 24 questions.  While it is not required by the FCC, in all cases the number of groups is exactly the same as the number of questions on the exam.

(The Canadian question pools are maintained by Industry Canada, and have a similar structure.)

Question pool	Effective dates	Topics	Groups	Questions on exam	Questions pool size
Technician	7/01/2010 - 6/30/2014	10	35	35	396
General	7/01/2011 - 6/30/2015	10	35	35	456
Extra	7/01/2012 - 6/30/2016	10	50	50	702
Canadian Basic	4/01/2007 – Present	8	100	100	964
Canadian Advanced	4/01/2007 – Present	7	50	50	545

The rules for generating exams only require that every exam have the same number of questions from each topic as there are groups in that topic, but the actual exams are always generated by taking one question, at random, from each group.

This creates an interesting phenomenon: questions in larger groups are underrepresented on the exams.  Consider the group, G7A (power supplies; transmitters and receivers; filters; schematic symbols).  With 24 questions in the group, each question in the group has a 1 chance in 24 of appearing on your exam.  Meanwhile, the group, T1A (basis and purpose of the Amateur Radio Service, penalties for unlicensed operation, other penalties, examinations), has only 10 questions, so each of those questions has 1 chance in 10.  While you will always get exactly one question from group T1A and one from G7A, any one specific question in group T1A is 2.4 times more likely to appear than any one specific question in group G7A.

Since the free online practice exams are generated in the same fashion as the actual exams, this phenomenon increases the number of practice exams you must take to have a high probability of seeing all the questions in the pool.

Assuming you decide to memorize the answers using random exams, just how many do you need to take?  The answer depends on how good your memory is and how confident you want to be in passing.  Some people are happy to squeak by on the second or third attempt, while others aren’t happy unless they score 100%.

Let’s assume your memory is pretty good, and you’re aiming for a score around 85%.  After all, you’re looking for the cheapest way to get your license, so you don’t want to risk paying the $15 exam fee (varies by VEC) again.

So you decide that you want to see 90% of the questions at least two times.  With a fairly good memory, as long as you cram most of your study into a few days immediately prior to the exam, you should be able to remember most of them.  Your long-term retention will be poor, but that doesn’t matter, as you’re just memorizing answers, anyway.

Because of the way the exams are generated, the easiest way to determine the solution was to write a computer program that generates many random exams using precisely the same rules used to generate the actual exams.  I averaged the results of 500 trials to eliminate the effects of random fluctuations.   For the mathematicians in the crowd, the result is a binomial distribution.

This table shows how many practice exams you need, and estimates how long it will take, assuming you spend 30 seconds per question:

Question pool	Number of exams	Questions per exam	Time per exam	Total "study" time
Technician	43	35	18 minutes	13 hours
General	56	35	18 minutes	16 hours
Extra	60	50	25 minutes	25 hours
Canadian Basic	40	100	50 minutes	33 hours
Canadian Advanced	41	50	25 minutes	17 hours

In all cases, when you reach the point where 90% of the questions have been seen at least twice, there are still approximately 2% of the questions which have been seen zero times and 8% which have been seen just once.  Unfortunately, at least 1/3 of the questions have been seen six or more times!

For those aiming for a higher score, practice exams are frustrating.  The closer you get to 100%, the more it repeats questions you’ve already learned.  To understand this, consider the case where you’ve memorized 90% of the answers.  Since practice exams pick questions at random, that should mean that 90% of the questions it picks are ones you’ve already memorized.

The exam groups exacerbate the problem, especially since the largest groups tend to have the hardest questions.  You take a 35-question practice exam, and you might only see 2 questions you haven’t yet memorized.  On average, you might see one new question every 5-10 minutes.

While these statistics are useful predictors of total effort, you don’t need to count the number of practice exams you take.  You’ll know you’re ready when you consistently pass them.

Some people aren’t happy unless they’re sure they will get 100% on the exam, so they ask us how many exams to be guaranteed to see all the questions in the pool at least twice.  Unfortunately, there is no number which absolutely guarantees it.   It’s a matter of probabilities.

It’s like flipping a coin many times.  Flip it once, and there’s a 50% chance you’ve only seen tails.  Twice, it’s a 25% chance.  Three times, it’s 12.5%.  It keeps going like that forever, never quite reaching zero.   After 100 flips, there’s still a .00000000000000000000000000008% chance you’ve only seen tails.  There is no finite number of coin flips that absolutely guarantees you will see at least one heads.

However, it is possible to determine the number of random exams you have to take to have a 90% probability of having seen every question at least twice:

Question pool	Number of exams	Questions per exam	Time per exam	Total "study" time
Technician	118	35	18 minutes	34 hours
General	189	35	18 minutes	55 hours
Extra	199	50	25 minutes	83 hours
Canadian Basic	120	100	50 minutes	100 hours
Canadian Advanced	117	50	25 minutes	49 hours

If you are going to use practice exams, we think HamTestOnline™ has the best. Here’s why:

• Ease of use – You’re going to be answering several thousand questions, so you want it to be easy. You don’t want to have to maneuver the mouse to click a tiny circle every time.
   
• Immediate feedback – While not knowing your score until the end more closely simulates the real exam, immediate feedback is better for achieving your memorization goal.   After you make a guess, you should see the correct answer while the question is still fresh in your mind.
   
• Save session – If you don't have time to finish the exam, you can sign in later, even from a different computer, and continue where you left off.
   
• Tracks history – It’s handy to be able to look back and review the questions you missed on prior exams.
   
• Integrated question pool display – A great way to accelerate memorization is to review the questions that give you trouble.