IAM masters/RoSPA gold?

[Horse]

Not chapter and verse, only subjective levels - see here. It's pretty comprehensive on the competencies, but the grading is subjective.

And there's the problem - at these rarified "I know it when I see it" levels, how do you identify and quantify differences?

e.g.:

Drive at a speed that is safe for the situation. Drive in a manner that is safe, smooth, systematic and making progress where it is appropriate whilst also showing consideration for others.

Overtaking manoeuvres must be well planned, carried out safely and within the speed limit.

Or . . .

You must be able to use a safe, systematic approach throughout to maintain the safety of yourself and other road users

You must be able to make progress in the traffic stream and overtake with consideration for other road users


Where's the differences between those two sets of standards?

[akirk]

And there's the problem - at these rarified "I know it when I see it" levels, how do you identify and quantify differences?

Isn't it similar to musicians (as an example) - it is relatively easy to test at grades 1-8 / diplomas / etc. - you either know your scales or you don't...
beyond that there is a whole progression of ability that becomes increasingly more competent and increasingly more difficult to assess...

you then get to the point of trying to compare Freddie Mercury with Pavarotti - both singers, or Keith Moon with Evelyn Glennie (they can both hit a drum)

possibly the more difficult it is to objectively assess, the more you 'know it when you see it' and the less it matters as the level is so good!

I am sure it is not a complete / direct analogy with driving, but I do feel that there are similarities...

Alasdair

[Horse]

The analogy perhaps falls down in that some singers can't sing - but produce good, listenable and enjoyable, music

And along those lines, a friend owns a pianola - if she pedals the thing fast enough, it plays quite complicated music, but neither have much skill!

[Horse]

Not chapter and verse, only subjective levels - see here. It's pretty comprehensive on the competencies, but the grading is subjective.

And there's the problem - at these rarified "I know it when I see it" levels, how do you identify and quantify differences.

A few things often stand out amongst those who are successful. Overtaking is one clear one - the ability to imagine that a potential opportunity is coming and being ready. The ability to just drive as yourself rather than trying to deliver a drive you think I might like to see (a big problem for me when in was on the other side of the process) is another.

Given what I do for a (partial) living I analyse more than others who go on gut feeling, but it's rare for us not to reach a consensus.

Indeed - the difficulty is writing it down in a way which it can be understood by someone else and used to make the same differentiation. Or maintaining conformity between candidates.

[RobC]

Even then the fleet driver industry wouldn't accept IAM F1rst as the advanced driving qualification because , Rospa gold has been the benchmark for many years...

Rob

Hi Rob,

I'm intrigued by the above quote. For what purpose (note: not reason) does the fleet driver industry accept... RoSPA Gold as the benchmark?

Thanks.

Hi Hir

Some of the fleet training providers eg http://www.theaa.com/aadrivetech/driver-training.html like their trainers to have a Rospa gold as part of the qualifications required for the purpose of assessing drivers. The main requirement/qualification would be to be a DVSA fleet registered ADI of a high grade which used to be a 5 or 6 but is now an 'A'

Rob

[GS]

Even then the fleet driver industry wouldn't accept IAM F1rst as the advanced driving qualification because , Rospa gold has been the benchmark for many years...

Rob

Hi Rob,

I'm intrigued by the above quote. For what purpose (note: not reason) does the fleet driver industry accept... RoSPA Gold as the benchmark?

Thanks.

I don't know if this is a purpose or reason but, in my experience, RoSPA is recognised as a company from which various courses and advice can be taken. Their driving tests is well thought of because it is graded and has to be regularly retaken. This gives drivers the opportunity to improve their grade over time.

I think it will be very interesting to see what, if any, difference to RoSPA the introduction of the AA's new Advanced Driving Diploma makes. This is also graded and 'has' to be retaken regularly. Some very large companies appear to be very interested in adopting this new test as an industry standard.

[akirk]

The analogy perhaps falls down in that some singers can't sing - but produce good, listenable and enjoyable, music

And along those lines, a friend owns a pianola - if she pedals the thing fast enough, it plays quite complicated music, but neither have much skill!

The analogy is starting to work better!
I can think of drivers who aren't 'by the book - technically good' but produce good drives
and in your second example are we not onto autonomous cars?!

Alasdair

[RobC]

And there's the problem - at these rarified "I know it when I see it" levels, how do you identify and quantify differences?

Isn't it similar to musicians (as an example) - it is relatively easy to test at grades 1-8 / diplomas / etc. - you either know your scales or you don't...
beyond that there is a whole progression of ability that becomes increasingly more competent and increasingly more difficult to assess...

you then get to the point of trying to compare Freddie Mercury with Pavarotti - both singers, or Keith Moon with Evelyn Glennie (they can both hit a drum)

possibly the more difficult it is to objectively assess, the more you 'know it when you see it' and the less it matters as the level is so good!

I am sure it is not a complete / direct analogy with driving, but I do feel that there are similarities...

Alasdair

Hi Alasdair

I agree that higher levels are more difficult to assess. If you take the IAM test with a 90% pass rate its not going to be difficult to assess a pass or fail. But if you have a level that only 10% achieve then you must have objective marking criteria which can quite easily be defined but is probably less easily interpreted by those who make the judgement.

I don't think we can compare Freddie Mercury and Pavrotti subjectively, not because their level is so good, but because their styles are so different so you would have to compare them objectively by criteria.
A subjective decision who was the best of the two could be down to personal taste or simply not having the knowledge or skills for example to assess opera singing.

Then again passing an objective test all depends on the criteria chosen. A driver who has never passed a test who can drive around Bombay unscathed in rush hour may have far more advanced skills to some than a Rospa gold driver

Rob

[jont]

The analogy perhaps falls down in that some singers can't sing - but produce good, listenable and enjoyable, music

And along those lines, a friend owns a pianola - if she pedals the thing fast enough, it plays quite complicated music, but neither have much skill!

The analogy is starting to work better!
I can think of drivers who aren't 'by the book - technically good' but produce good drives
and in your second example are we not onto autonomous cars?!

Or just modern cars where the driver doesn't appreciate/realise how much the electronics is doing for them behind the scenes?

[RobC]

The analogy perhaps falls down in that some singers can't sing - but produce good, listenable and enjoyable, music

And along those lines, a friend owns a pianola - if she pedals the thing fast enough, it plays quite complicated music, but neither have much skill!

The analogy is starting to work better!
I can think of drivers who aren't 'by the book - technically good' but produce good drives
and in your second example are we not onto autonomous cars?!

Or just modern cars where the driver doesn't appreciate/realise how much the electronics is doing for them behind the scenes?

Good point Jon. Modern cars flatter the driver with their safety features and ease of manoeuvring and you don't even need to be able to change gear, you could pass an advanced test at a high level in an automatic.

[akirk]

And there's the problem - at these rarified "I know it when I see it" levels, how do you identify and quantify differences?

Isn't it similar to musicians (as an example) - it is relatively easy to test at grades 1-8 / diplomas / etc. - you either know your scales or you don't...
beyond that there is a whole progression of ability that becomes increasingly more competent and increasingly more difficult to assess...

you then get to the point of trying to compare Freddie Mercury with Pavarotti - both singers, or Keith Moon with Evelyn Glennie (they can both hit a drum)

possibly the more difficult it is to objectively assess, the more you 'know it when you see it' and the less it matters as the level is so good!

I am sure it is not a complete / direct analogy with driving, but I do feel that there are similarities...

Alasdair

Hi Alasdair

I agree that higher levels are more difficult to assess. If you take the IAM test with a 90% pass rate its not going to be difficult to assess a pass or fail. But if you have a level that only 10% achieve then you must have objective marking criteria which can quite easily be defined but is probably less easily interpreted by those who make the judgement.

I don't think we can compare Freddie Mercury and Pavrotti subjectively, not because their level is so good, but because their styles are so different so you would have to compare them objectively by criteria.
A subjective decision who was the best of the two could be down to personal taste or simply not having the knowledge or skills for example to assess opera singing.

Then again passing an objective test all depends on the criteria chosen. A driver who has never passed a test who can drive around Bombay unscathed in rush hour may have far more advanced skills to some than a Rospa gold driver

Rob

Hi Rob, I think that there is a logic disconnect here - you seem to feel that a 90% pass rate indicates a low level of acheivement / not being objectively marked - you can't conclude that without understanding the candidates' ability - it is more likely that it is simply that they are well prepared and ready for the exam - a test is simply an observation to formalise the understanding of where someone is / their level - ideally if tthe teaching / preparation is good and the candidates capable you would expect a 100% pass rate - if elsewhere you have only a 10% pass rate then that suggests the possibility that candidates are not ready / not capable / not prepared...

also, I think it is important to realise that ROSPA Gold, while indicative of good driving is not the most advanced / does not suggest that top level of driving - there are levels of driving so far beyond that most candidates for the Gold wouldn't even begin to imagine tthe possibility... and to suggest that only testing objectively is to miss out on so much...

we all know that an 18 year old with a customer service NVQ is not as capable / qualified as their manager who has spent 30 years in the job and knows evey nuance of how to deal with customers - yet in your philosophy he would be more capable - this is a very weird way of thinking and sadly indicative of so much today where the ability to 'prove' ability on paper is considered more important by some than actually being able to do it...

so yes, ROSPA Gold is good - but you need to open your eyes to what starts where it finishes - I understand that corporates like the sound of ROSPA Gold / the retesting is no doubt good / a piece of paper allows them to CYA - but to suggest that it is the pinnacle of AD simply because it is objectively tested / and then retested is of course not accurate - come and join my new qualification - I will objectively test your ability to drive in xyz manner and I will insist on a retest every year - it must therefore be better than the ROSPA Gold / very advanced of course we know that is silly, so is the lack of acceptance of abilities beyond those that can be tested objectively...

Pavarotti and Mercury are good examples - you try testing either objectively, there isn't an appropriate music qualification - yet they are clearly way beyond the ability of someone with grade 8 singing - in every discipline, PhD work is not and can not be tested simply objectively - it is peer assessed and tested against other known levels to judge - so with driving, there comes a point where a driver can't simply be assessed against a set of objectives -but must be peer assessed - their driving is too good for the objective tests...

We shouldn't see that as a threat to ROSPA Gold drivers - it isn't, it should be inspirational, the Gold is a high level of driver and a stage on a journey which has even more to offer...

Alasdair

[Mr Cholmondeley-Warner]

I would hope no more easily than in a manual. The candidate will still be required to show that they understand and can use the gearbox to their advantage, whatever type it happens to be.

I fear it is easier, however much the test tries to level the playing field. The candidate has to know when to override the gearbox, such as when going downhill, or perhaps to maintain speed in a 30 limit, but they never have to actually change gear. Even, were they to use manual override all the time (which would be frowned upon by most examiners), they would never have a clutch to control, or revs to match. In most cases they can just forget gears, and drive as if the system had only 3 vehicle control phases.

[Mr Cholmondeley-Warner]

Lots of stuff, and:

you seem to feel that a 90% pass rate indicates a low level of acheivement / not being objectively marked - you can't conclude that without understanding the candidates' ability - it is more likely[1] that it is simply that they are well prepared and ready for the exam - a test is simply an observation to formalise the understanding of where someone is / their level[2] - ideally if the teaching / preparation is good and the candidates capable you would expect a 100% pass rate - if elsewhere you have only a 10% pass rate then that suggests the possibility that candidates are not ready / not capable / not prepared...

I would argue that 90% DOES indicate quite an easy test. Not for nothing is the pass rate used as a guide to varying the difficulty of school exam questions. Ideally you'd expect a 100% pass rate there, too, but we know that will never happen.

And to answer the parts in italics:

[1] I would argue very strongly that it's NOT more likely, statistically. Worthy fellows though IAM observers are, to suggest that they uniformly prepare their candidates so well that 90% pass a very difficult test is statistically much less likely than the suggestion that it's actually quite an easy test in the first place
[2] if you only have a pass / fail criteria, you don't actually know levels, only a (rather low) threshold. (and Martin, yes I'm remembering F1rst )

and:

I understand that corporates like the sound of ROSPA Gold / the retesting is no doubt good / a piece of paper allows them to CYA - but to suggest that it is the pinnacle of AD simply because it is objectively tested / and then retested is of course not accurate

Read his posts. He never said "pinnacle". He said "benchmark". They're about as different as your Mercury and Pavarotti analogy.

EFA: [1] and [2] were back to front.

[RobC]

Hi Alasdair

I didn't say that Rospa gold is the pinnacle of achievement, it certainly isn't it is only one measure of driving ability there are many others including some courses I deliver which are the ones companies are willing to pay for rather than advanced qualifications that individuals require.

I agree that pass rates may also indicate test readiness however the low pass rate of say an IAM F1rst also indicates a level of achievement not attainable by all.
My philosophy is not that an 18 year old would be more capable than someone with 30 years experience in a job and I pointed out earlier that an examiner who gives a 19 year old with a years experience an IAM F1rst while never having a Rospa gold member achieve a F1rst is not what I would normally expect. A test is only indicative of the performance of the individual over an hour. A driver with 30 years experience has many times the unseen experience of a 19 year old on test.

Rob

[RobC]

Lots of stuff, and:

you seem to feel that a 90% pass rate indicates a low level of acheivement / not being objectively marked - you can't conclude that without understanding the candidates' ability - it is more likely[1] that it is simply that they are well prepared and ready for the exam - a test is simply an observation to formalise the understanding of where someone is / their level[2] - ideally if the teaching / preparation is good and the candidates capable you would expect a 100% pass rate - if elsewhere you have only a 10% pass rate then that suggests the possibility that candidates are not ready / not capable / not prepared...

I would argue that 90% DOES indicate quite an easy test. Not for nothing is the pass rate used as a guide to varying the difficulty of school exam questions. Ideally you'd expect a 100% pass rate there, too, but we know that will never happen.

And to answer the parts in italics:
[1] if you only have a pass / fail criteria, you don't actually know levels, only a (rather low) threshold. (and Martin, yes I'm remembering F1rst )
[2] I would argue very strongly that it's NOT more likely, statistically. Worthy fellows though IAM observers are, to suggest that they uniformly prepare their candidates so well that 90% pass a very difficult test is statistically much less likely than the suggestion that it's actually quite an easy test in the first place

and:

I understand that corporates like the sound of ROSPA Gold / the retesting is no doubt good / a piece of paper allows them to CYA - but to suggest that it is the pinnacle of AD simply because it is objectively tested / and then retested is of course not accurate

Read his posts. He never said "pinnacle". He said "benchmark". They're about as different as your Mercury and Pavarotti analogy.

Agree fully with Nicks comments above!