Software Developers In Test: - PowerPoint Presentation

Slide1

Software Developers In Test: The Supply-Side Crisis Facing Agile Adopters_____________A reality check on where we areReflections on how we got hereA discussion on scaling supply

Facilitated ByRichard Neevewww.richardneeve.net

UK Test Management Forum

Balls Brothers, London

30

th

October

2013Slide2

Some WarningsThe evidence herein is unashamedly anecdotal.Looking to work with Paul Gerrard and Graham Thomas on a survey for review at the next Test Management Summit

.If you’re opposed to agile or using SDITs, that’s fine but you may not get much/anything from this session as I’m running this on the premise that we need to adequately fulfil the market demand regardless of its merits.There may not be any realistic answers.But any answers are probably in this room. I doubt anyone beyond testing will solve this for us. So if ever there was a topic that relied on the group’s participation and wider community influence that is mentioned in the standard TMF introduction, this is it!We won’t crack this today; I would like us periodically re-visit this topic to gauge progress (hopefully in a more evidence-based fashion).Slide3

Definitions (1 of 2)If you disagree, please indulge me as I don’t want to get bogged down here.SDIT:“Software Developer In Test”.A (more?) common alternative label is SDET (Software Development Engineer in Test). Others? Code-infected tester?

Surprising paucity of succinct definitions offered online but we’re basically talking about people who can:Write automation frameworks.Develop automated test cases.Use their coding skills to pair with a developer in TDD (for example).Independently develop non-trivial bespoke test tools as required.Conceivably produce production code if that’s deemed appropriate.Potential ‘Litmus Test’ for future debate: “Someone that could, in very short order, become viable in a full commercial dev role if they were so inclined”?Slide4

Definitions (2 of 2)Crisis:“A stage in a sequence of events at which the trend of all future events, especially for better or for worse, is determined; turning point”. (Source: www.dictionary.com) Seems justified to me. May or may not represent an existential threat to testing. I think it’s a top 3 challenge for testing (perhaps no.1). Large opportunity cost already incurred? Could get a lot better/worse depending on whether/how it’s addressed.Slide5

Where are we?Slide6

The Broad Macro PictureAgile is now mainstream and calls for the blurring of test and dev roles. So demand for SDITs is broad-based and rapidly accelerating as agile continues to spread and mature.Lots of warning (min 5yrs, arguably 10+yrs?) but we haven’t/aren’t geared up to supply SDITs in volume so the market has been caught flat-footed and appears to be in serious under-supply.Existing limited supply is ‘tightly held’ hampering market liquidity. SDITs are typically looked after well (financially and otherwise). Clients increasingly realise backfilling will be painful, uncertain and could easily take longer than the incumbent’s notice clause (jeopardising knowledge transfer).Slide7

Demand Under-pinned By ConvictionWhere SDIT demand exists, it seems very ‘committed’. I’m sure that’s true of other roles but it seems that:Once an environment philosophically commits to the SDIT concept, it really wants them and they’re rarely seen as a ‘nice to have’ thereafter.A contrast with so many environments where the test manager has to take what they can get through voluntary automation efforts post 6pm.SDITs are seen as key to achieving ‘sustainable agility’ through automation enabling Continuous Integration and the avoidance of lapsing into ‘Wagile’ style test cycles.

SDITs are seen to speak directly to the “Shift Left” narrative of reducing time/cost (and can enjoy strong senior sponsorship as a consequence).Observation: I’ve seen/heard ‘Shift Left’ used more in the last month than I have in the last decade.SDIT demand is compounded by/with what appears to be similarly committed (and growing) demand for Behaviour Driven Development (BDD) and

Test Driven Development (TDD)

. This ‘slip-streaming’ demand effect seems perfectly logical.Slide8

The Political/HR DimensionIn reality, implementing the agile evangelist’s vision in which everyone swaps between test and dev activity as required, is very hard since:Many developers won’t do testing (beyond the unit testing mandated by the dev job market).Many testers don’t have comprehensive development skills.I’ve seen devs threaten constructive dismissal claims over this as they see it as a diminution of their role (and maybe it is).

I’ve seen testers resign (un-prompted) in fear of having to skill up in order to ‘go under the hood’.These are very delicate matters, especially when you’re dealing with permanent staff.Are agile evangelists/coaches living in a bubble to some degree? Are they disproportionately:Surrounded by self-selecting enthusiasts at the top end of the ability range as they move from pilot to pilot?Exposed to highly innovative start-ups and small companies that are highly progressive in their thinking and perhaps inherently have an “all hands to the pump” mentality?Shielded from the broader range of attitudes/concerns faced by those who have to scale the agile adoption across the wider organisation once their agile coach’s involvement is winding down or concluded?

If agile evangelists/coaches are setting undeliverable expectations, are they inadvertently causing part of the problem and if so how might they be part of the solution?

Either way, these factors increase the appeal of those willing and able to bridge dev and test.Slide9

Tales Of EscapeYou may have been active in SDIT recruitment but not impacted (yet) if:You’re acting on behalf of a tier 1 investment bank (or similarly price-insensitive organisation) that can aggressively bid for resources.You’re acting on behalf of a firm like Google or Microsoft that can attract people through money, high profile innovation that even my mum may have heard of and an environment with all the trimmings (e.g. pool tables, free food, consoles, etc). You’ve outperformed what you’re offering having lucked into one or more appointments because the candidate(s) had particular reasons to join your team (e.g. they live around the corner and have just had a new baby).

You haven’t had to go to the external market having successfully tapped an internal supply of willing or conscriptable resources.You’ve successfully gambled on a ‘wildcard’ appointment (e.g. a cheap graduate or taking someone from a tangentially related function e.g. tech support).Slide10

The Limits Of Those TalesThat’s all fine but:On current trends even clients with a strong market position may soon start to struggle.Everyone needs a bit of luck but it’s not a strategy in itself and is finite.Individual anecdotes about successfully using internal/wildcard candidates are great on a micro level, but they’re not going to sate the breadth of demand at the macro level.What if you’re hiring an SDIT for a relatively unknown and somewhat unlucky SME that’s:

On a trading estate in, say, StainesClose enough to the city to have to compete with Goldman Sachs et alClose enough to the M4 corridor to have to complete with MS et alWorking to pay scales set by someone living in dreamland?Slide11

The Market Pricing Impact(experienced committed demand) + (new potentially ill-informed demand) + (limited illiquid supply) + (wide variations in price sensitivity) = impaired price discovery …leading to extreme pricing and a dysfunctional market that’s struggling to clearOr to put it in less abstract terms…I keep seeing SDIT job adverts, sometimes side-by-side in my Linkedin feed, that are comparable in specification, sector and location, paying basics of £35k and £75k.

Former probably reflects ignorance about the SDIT market and prejudices (perhaps baked into formal pay grades) about the value of functional testers. Latter probably reflects an experienced, committed and price-insensitive client.But who knows? Contract rates for SDITs seem to exhibit similar disparities.Audience: Any insights on the basic salaries/day rates at which most deals are being done in London, both outside and inside of finance? How long are vacancies typically open for? Any prospective SDIT hirers shocked at what they’re hearing?Slide12

Deeper ‘Trouble’ Ahead?I have the Bloomberg mobile app with alert subscriptions (10 to 15 a day).I’ve had hundreds of positive alerts in recent months with literally just 3 or 4 negative exceptions:Earnings Per Share for company X beats analysts’ estimates by Y.Confidence index X at expected level + Y.Unemployment in country X fell by Y more than expected.

Etc. It has been notably relentless, for months.ONS data for Q4 2012 showed cash held by corporates other than banks was £318bn. FTSE100 cash reserves in 2013 are at £74bn, up from £12bn in 2008. Excessive cash reserves often aren’t favoured by investors (e.g. the demise of GEC Marconi) especially in this low interest rate environment.Are board room execs also watching these positive metrics? Are they collectively about to push a lot of green-coloured buttons?Good to recognise when problems are ‘nice to have’. But are we about to get punished (at least in opportunity cost terms) for not skilling up while more projects were on ice?

Homework: Read up on ‘

elasticity of supply

’ and consider its implications for this topic.Slide13

A Quick Re-CapBroadening/accelerating/committed demand set against limited/illiquid supply.So basically we’re all fishing in the same puddle for resources considered to be pivotal. Can we at least turn the puddle into a small stream?Indications of a dysfunctional market.Some unaware (through not having to go to market, or luck, or having a strong market position).Some very aware through painful experience.Questionable preparedness for a large wave of capex by corporates (if it happens).

There are technical, financial and political factors in play.I think this is a huge challenge for the next economic cycle.Slide14

How did we get here?Slide15

Causal Factors: My Perspective (1 of 3)My analysis is flawed as it only dates back to 1998 but for what it’s worth:Historical Structural Separation Between Dev And TestThe ‘test independence’ agenda was firmly in focus when I started; seemingly less so now. Y2k an catalyst? Not saying independence is bad but the market now seems to accept a risk of reduced independence as a fair price for closer collaboration.

Many strongly embraced the idea of test being an arms-length ‘formal customer’ of dev; probably a by-product of the test independence agenda. This suited lots of people including tool vendors.The ‘professionalisation’ of testing (manifested via ever-burgeoning standards, certifications, tooling, community, vocabulary, etc) has perhaps encouraged differentiation – and therefore an increased degree of separation – from dev, even though there’s probably no intrinsic reason why that should be so.The ‘equal pay for testers’ lobby, which appears to have failed, seemed to ‘dig in’ quite deep and this may ultimately have accentuated any separation.Slide16

Causal Factors: My Perspective (2 of 3)Apparent long-term downward trend in the technical skills of candidatesReported by many with a long-standing involvement in technical recruitment (me included).I don’t want to get political here, but is this the downstream impact of ‘A’ Level Mathematics and Computer Science being diluted, causing universities to react in sympathy to secure income following cuts in central government funding?Testing’s initial reaction to

The Agile Manifesto seemed somewhat delayed/passiveThe Agile Manifesto isn’t the origin of agile but it seemed to be the trigger that made it mainstream thinking among a whole generation of people like me.I’m not sure we came out of the blocks quickly enough on the whole agile testing piece. I often felt there was a victim mentality among testers. We seemed unsure about how/whether we fitted in which makes me doubt whether we fully exploited the opportunity to provide thought leadership in this area.By the time the manifesto was published, many testing specialists were carrying some very negative baggage regarding automation. I think many of us were unduly reticent about selling automation again and using our automation skills/experience as a vehicle to engage in the wider process.These factors perhaps created a sense that we weren’t leading the agile testing skills agenda and by extension we weren’t collectively skilling up in line with the emerging market requirements (even though some clever individuals were). Slide17

Causal Factors: My Perspective (3 of 3)Unfavourable career path dynamicsDevs don’t want to move into test as they see it as a diminution in their role and status.If you’re seriously technical, your career prospects are probably better in dev (since it seems more common for senior managers seem to be drawn from dev rather than test).Quoting current SDIT rates often elicits a response along the lines of “we can get a dev for that” so you don’t get far before hitting political constraints on SDIT pay, which impacts the demand signals being emitted in the market.

Scarcity of seriously technical test specialists has made succession planning difficult so ambitious people are wary of getting stuck in their career.Many people tire of the constant treadmill of learning the latest tech and want to step off it.In most cases financial progression seems to necessitate going the hands-off management route - especially for permanent staff. Again I don’t want to get political here, but over the last 10 years or so, property price inflation has been a big issue for people in their 20s and 30s (i.e. those on the cusp of management).Recasting organisations around multi—disciplinary and product-centric teams has sometimes seen the reduction/removal of the test management layer. That may or may not be the best thing, but where it has happened it has often:

Reduced the advocacy for the training and development of test specialists.

Reduced career opportunities for testers who are aspiring test managers.

Resulted in people like me being asked to return to hands-on work which generally isn’t attractive.

Placed increased demands on the soft skills of testers and created circumstances where the tester at the coal face is the most senior testing specialist in the organisation. Many testers have no appetite for the politics from which they have previously been shielded and have acted accordingly.Slide18

How can we scale SDIT supply?Slide19

Or Is Doing Nothing An Option?Opportunity cost implications?Look enormous to me (w.r.t both influence & revenue)Do we have bigger fish to fry?With the possible exception of the offshoring nightmare it’s hard to think of many (any?) compelling candidates.Existential threat from eventual irrelevance? Is it too late or is the problem simply intractable? Are we so behind the curve that it’s a rear guard action from here on in?

Maybe but the situation still feels redeemable.Slide20

Tactical View: Where Are They Now?In the short term and on the margins, clients may benefit from being more informed/proactive in knowing where the existing supply can be accessed: Events that are disproportionately likely to attract the interest of SDITs such as: LTG, Skills Matter seminars,

eXtreme Tuesday Club, Meetups for SDITs, etc.Focused Linkedin groups like this one.Specialist recruitment agents.Others?Bonus Poaching Tip: I’ve noticed media organisations seem to do well in recruiting SDITs.On a selfish tactical level this may help some individual clients, but it doesn’t create fresh supply in the market as a whole.Slide21

Strategic View: Some Systemic OptionsRolling back market norms?Paying to train contractors?Tackling specificity of job specs?Restructuring pay grades so people don’t have to go the management route to progress financially?Taking more risks on graduates?More use of propositions like

FDM and Novus? Deeper links with universities?Changing contractual arrangements for recruitment?Tighter PSLs (carrot)? Looser PSLs (stick)?Can we do anything different commercially to encourage risk taking? Changing the risk/reward balance?Exploring insights from recruiters?

Any discernible themes in the reject pile? Are there risks in there that we could ultimately manage as we do in other areas? How would we need to flex to turn the top, say, 10% of rejections into placements?

Any scope to skill-up non-techies or de-skill the SDIT role without impacting its effectiveness?

Anything around tooling?

Converting script readers to script writers? A matter of building confidence through coaching?

Can we cut demand too?

Any scope to cannibalise the SDIT remit across other, more readily filled, roles?

What might it take to make more (pure) developers want to do this kind of work?Slide22

SummaryThe SDIT demand profile is broadening/accelerating/committed and set against limited/illiquid supply, provoking symptoms of market dysfunction.Recruitment consultants and direct hirers seem to be increasingly desperate.Questionable preparedness for a large wave of capex by corporates (if it happens).Status quo looks unsustainable to me.A key challenge - both present and future – and not just for testing.

The dynamics involved are numerous, complex and inter-connected. They are financial, technical and political; some having origins dating back many years.Some clients are unaware (through not having to go to market, or luck, or having a strong market position). On the current trajectory, even those with a strong market position may soon start to struggle.On the margins, clients and their recruitment agents may be able to identify new sources of existing candidates by being better informed and more proactive. But taking a market-wide view, this is just ‘robbing Peter to pay Paul’.Genuinely and sustainably freeing up supply in a way that will benefit the broader market will require time, innovation and risk-taking (if it happens at all).Slide23

But In The Meantime…Thank you for listening and participating.I’ll create a blog entry on www.richardneeve.net about this session so we can continue the discussion in the comments section.The slides will also be on uktmf.comI‘m on

Linkedin.Email: richard@richardneeve.net