Mehdi Khalili

Product launch checklist

Sun, 12 Aug 2018 00:00:00 +0000

A few months ago, I was talking with a client about things they should consider before launching a new product we, Readify, were helping them to build. I put a list together and walked them through it.

I thought I’d document the checklist here for future reference. This by no means is an exhaustive list of everything you need to consider; but rather a list of things I’ve seen commonly overlooked. Then again, there is a lot in here. Just cherrypick what’s applicable to your particular case.

I am assuming you already know how to deliver quality software. With that out of the way, here’s the list.

User Experience

Before doing anything else, identify and validate the problem you’re working on. Design Thinking is your friend here.
Perform user/usability testing, as you start creating prototypes and develop features. I’ve had good experience with User Testing. A few things to consider for usability testing:
- Who to test the product with?
- Developing personas for usability testing (and early adopters)
- How and where do you find users?
- Prepare the usability scripts and scenarios
- Arrange and run interviews
- What to feed back into the roadmap and what to discount
- Do you need confidentiality agreement for external user testing?

Product Management

Use a sound product market fit process and strategy. I am a big fan of Lean Startup.
Perform a competitor analysis to see what others are doing.
Are you doing soft launch or big bang release? Do soft launch whenever possible.
Create and maintain a product roadmap. Refine the roadmap based on market analysis as well as the feedback from usability testing and early adopters.
Do you publish your roadmap? How far into the future? Don’t commit to long term roadmap. Your roadmap will change by the time you get there.
How do you gather feedback from your users? How do you prioritise their feedback along new feature development?
What needs to be in place before you release the product to early adopters? What needs to be in place before you open the flood gates?

Marketing

Find a name! Sounds kinda funny; but a lot of people fail at this. A few considerations:
- Brainstorm around possible names
- Google them to make sure they don’t point to existing products and services or to bad press
- Make sure the name you pick is google-able; e.g. don’t name your product go
- Make sure there are some domain available for that name
Buy your domain and consider buying a few other top layer domain names for your domain; e.g. .com, .com.au, .net, .io.
Define a tagline for the product.
Define a pricing structure and strategy. Test your pricing structure with your early adopters. Do you provide a different pricing for your early adopters?
Think about marketing. Do you need a landing page? Who’s going to write the copy? Who’s going to design the landing page? Who’s going to do SEO?
If you don’t have this already, design a clear brand identity; i.e. colour palettes, icons, style guides etc.
Do you want to have a launch “party”? Do you know your launch date?
Consider a launch plan. Are there magazines, blogs or publications you want to target to promote the launch of your product? Have your articles, tweets etc ready in advance.
Do you need a social media presence?
- What’s the social media persona you want to embody?
- Who’s going to be the social media person?
- Who’s going to write tweets and updates?
- Do you need an approval process for new updates and responses?
- Is the twitter handle available?

Business Processes

How are you going to sell your product? Where can your customers find your product?
Can you accept payment from your customers?
Who’s going to provide application support for end users and how? Phone support, in-person, in-app chat, Facebook, Twitter etc?
Is your product used globally? If yes, (how) will you provide round the clock support?
How do you train your support staff with upcoming features?
How will you communicate changes and new features to your users?
Do you need to develop help, documentation or FAQ for the product? Try to aim for a product that’s so user friendly it doesn’t need a lot of support or training.
Do you have a CRM? Does it fit this product?
Create privacy agreements for your users
Create license agreement for your users/customers

Security

What kind of data would you be storing - PII, PCI, PHI?
What sort of regulations do you need to comply with - APP, GDPR, IRAP, NEHTA etc?
Do you have data sovereignty requirements?
Consider creating a responsible disclosure policy/guideline. Checkout Tesla’s for inspiration.
Consider bug bounty programs; e.g. bugcrowd.
Create a data retention and destruction policy.
Create a data breach response policy.
Consider creating a BCP and DR policy.
Do you have a security monitoring and alerting system in place?
Consider performing penetration testing.
- Do some research and find reputable pen testers.
- Do you want to do blackbox, greybox or whitebox testing?
- How often would you redo the test?
- Does the vendor allow free/discounted retest if they find vulnerabilities? If yes, what’s the grace period?

Technical

Do you have a logging, monitoring and alerting system?
Do you have a sound backup strategy in place? Do you test your backups on a regular basis?
Do your BCP and DR policy cover your technical recovery? What’s your mean time to recovery?
Have you done load and stress testing on your app? Can you monitor your application performance proactively?
Do you have health dashboards setup?
Can you do zero downtime release? This is much more critical if you are working on a globally used product with no “after hours”.
What’s your SLA? Do you have maintenance window? Is it communicated in the license agreement?

What have I missed? What do you take into account for a product launch?

Reinventing myself

Sun, 05 Aug 2018 00:00:00 +0000

One night, as I was laying on my bed in early 2018 thinking about the purpose of life, a question came to mind: “How will I feel at my death bed? What regrets will I have?”. This led me down a path of soul searching and self discovery for several months, trying to make sense of who I really am and what I really want in life.

This is the outcome of that journey, my personal mission statement, what I stand for and what I want to accomplish in my life.

Mission Statement

I live a life of contribution, with integrity and empathy, and I take full responsibility for my health, happiness, relationships and career.

When I die, I want to be remembered by family, colleagues and friends as a trustworthy caring person who had a lasting positive impact on their lives and the lives of others.

Principles

I centre my life and every decision I make by following these principles:

Integrity: I honour my word.
Empathy: I seek to understand others by seeing the world through their feelings, thoughts and experiences.
Courage: I say and do what I feel is right, even when it’s risky to do so.
Humility: I don’t let my ego control me or hurt others.
Accountability: I take complete ownership of my life and what happens to and around me.

Personal development practices

Here is a few practices that I will try and embed in my life to enable me and my mission:

Exercise regularly
Spend quality time with my wife and kids
Call three friends every week
Cultivate an attitude of mindfulless and gratitude
Read and write regularly

Listen, Challenge, Commit!

Sat, 23 Jun 2018 00:00:00 +0000

Listen wholeheartedly, challenge others with respect, commit to a decision, regardless of whether you agree with it or not, and execute on that commitment. This simple decision making process helps you build highly productive and cohesive teams.

Listen

“Most people do not listen with the intent to understand; they listen with the intent to reply.” - Stephen R. Covey

And some people don’t even listen - they either completely tune out as the other person speaks or constantly interrupt them!

To create a productive team, the first and most important step is to listen. When I say listen, I don’t mean to hear the other person. I mean listen to them with empathy. Put yourself in their shoes and see the world from their point of view, be genuinely curious and ask questions to understand their feelings, pain points and goals.

A lot of arguments happen because people don’t listen wholeheartedly to others. It is easy to get defensive and argumentative when you feel the other person doesn’t care about your idea. So it’s very critical that during this phase you don’t interrupt, disagree, or even agree. You do one and only one thing: you listen intently.

Challenge

“The ultimate measure of a man is not where he stands in moments of comfort and convenience, but where he stands at times of challenge and controversy.” - Martin Luther King, Jr.

Challenging, as awkward and uncomfortable it might be, is critical to make a good decision, grow a team, and create cohesion and trust between team members. Without challenging, you may make a wrong decision, or even if the decision is right, you may not buy into it without fully understanding its details.

So after everyone’s had a turn in presenting their ideas, you move to the second step - Challenge. Make this transition very clear and explicit. Up to this point, no one was allowed to agree, disagree or argue with others. This is your chance to challenge others. The goal is not to win an argument. The goal is to build on each others’ ideas or to explain your view and get buy in. So don’t hold anything back in this phase. If you disagree with someone, explain your objection clearly and respectfully.

People repeat themselves over and over again when they don’t feel heard or understood, and challenging, without listening, could turn into a heated argument. So everything I said above about listening applies just as much here. When someone’s challenging an idea, everyone needs to tune in, listen to them intently, empathise with their objective, fears and uncertainties about the idea.

Don’t interrupt the flow, don’t get defensive, don’t cut conversations short and don’t hold anything back. It’s a lot cheaper to make a right decision everyone buys into than to make a rushed one. At the same time, don’t go around the circle or on tangents. Move to the next step once everyone’s raised their objections and explained their ideas.

Commit

“When you’re surrounded by people who share a passionate commitment around a common purpose, anything is possible.” - Howard Schultz

Once you’ve made a decision, regardless of who agreed or disagreed with it, everyone should commit and execute it like it’s their own decision. There is no argument, no challenge, no “but I didn’t agree”. Everyone has fully discussed their ideas and challenged others. You are heard and understood. The only thing that matters now is the decision made by the team. Roll up your sleeves and help your team execute on that idea with the best of your ability.

Not every decision can be made with involvement from everyone though. Many times, particularly in larger organisations, you find yourself executing a decision made by another group. Now what?!

You have two choices: you can fully commit to the decision and execute it as if it was yours, or you can choose not to commit. Let’s break this down and see it in action:

It’s a great decision and you commit: it can’t get any better than this.
It’s a great decision and you don’t commit: you just obstructed your team and organisation from achieving its full potential. A decision or idea, no matter how great, is doomed to fail without proper commitment and execution.
It’s a bad decision and you don’t commit: without commitment you don’t invest much effort into it. You waste a lot of time without achieving much; so no one, including your boss, buys into your arguments around why it’s a bad idea, and if the decision fails, you might be blamed for it because of your lack of commitment.
It’s a bad decision and you commit: you show your full commitment and give it your best shot even though you think it’s a bad decision. With full commitment, you can try multiple approaches and come to an objective conclusion. With your full commitment, what appeared to be a bad decision might actually turn out to be a great one. Even if it doesn’t, when you go to your boss and say “hey, I’ve tried these approaches and this doesn’t seem to work. I don’t think this is a good idea”, you have much higher chance of being heard.

You see - it doesn’t matter if the decision you’re handed seems to be a good or a bad one. Everyone wins when you commit and everyone loses when you don’t. Commitment is the difference between doers and whiners, achievers and victims, great teams and average ones.

A team is not a team without commitment - it’s a collection of individuals with personal agendas. We’ve all worked in those teams - they’re not fun!

Teams with no ownership

I know quite a few managers and executives who complain about lack of ownership and commitment from their team! In fact I used to be one of them.

Most people don’t wake up in the morning aspiring to be a blocker or slacker. They want to do something they like and value. The challenge is that it’s hard to commit when your ideas are not valued and your voice is not heard.

A lot of companies move the decisions to where the information is: to higher management. Ironically, your chances of making a good decision is quite limited if you’re not involved in day to day execution of it. If you want commitment, share the information with those who should execute the decision and/or live with its consequences, listen to them, challenge and coach them, and involve them in the decision making process. I am absolutely certain you will get much more commitment from your team when you do this.

Listen, Challenge, Commit

“Leadership is the art of giving people a platform for spreading ideas that work.” – Seth Godin

Listen, Challenge, Commit is a very simple decision making process for creating great teams. The process, while simple, is not easy to implement: most of us are really bad at listening, we find challenging awkward and uncomfortable, and don’t fully honour our words.

You might find it hard to start with this; but I’m certain you will get better the more you try it. One thing that certainly helps implement such a process is communicating your values and expectations clearly with your team. Here is how I communicated my values and expectations with a team I led a while back. As you see, there is a lot of alignment between this process and my values, and that made it somewhat easier.

I hope this helps. Leave me a comment and let me know what you think and please let me know how you go if you decide to give this a shot. I’m also keen to hear about your approach to decision making and execution, if you have a different process.

Position descriptions are harmful

Tue, 18 Jul 2017 00:00:00 +0000

Companies go through so much pain to hire the best and brightest (at least good ones do) and when they find the one after spending countless hours interviewing many, they give them a position description that can be more or less translated to “We don’t really care what value you can add to our business. All we really expect from you is the following bullet points.”. This is not all on employers. A lot of employees demand a position description if not given one!

Here’s a few questions I’ve heard and conversations I’ve had over the years.

How can I do performance reviews without a position description?

You should NOT do performance reviews. Performance reviews are negative, intimidating and harmful. What you should do instead is regular informal catch ups with your team. We are humans. We don’t need an annual process. We can talk.

When you’re unhappy with the performance of a team member, you shouldn’t wait a year to tell them. As you wait longer, the under performance or the bad behaviour rubs on you; so you get (passive) aggressive towards them, and that becomes a mutual interaction over time which makes them and you less happy and productive. Instead talk with them as soon as you see a problem and work with them to fix it.

Likewise, when a team member does something great, don’t wait a year to tell them. Tell them straightaway so they know their good work is seen and acknowledged.

But how can I do pay reviews if I am not doing performance reviews?

One thing that is worse than annual performance reviews is tying salary review to it.

Without a position description, I don’t know what my manager expects from me!

Really?! When was the last time you read your PD? More importantly, when was the last time you talked to your manager and asked her how you are going and if there are things you can improve on?

Do NOT replace regular human interactions with a document with a few bullet point on it. If you like your company and want to still work there in a few years, do NOT use your PD as the basis for what you should and shouldn’t do. Ask your manager and peers for feedback, frequently.

We have position descriptions and our employees are very passionate, committed and creative

Great. Putting position description aside for a second, few companies can maintain passion, commitment and creativity in their employees. So, well done.

Getting back to position description, I have written plenty of them and in my experience they are very time consuming to write and maintain. Requiring that much effort and time, I would love to hear what value you, your boss or your employees are getting from them and if you could do the same or better without them. Please leave me a comment.

How can I hire without a position description?

This is a large topic that deserves its own post. In a nutshell, instead of selling a position by describing it, you should sell your vision, culture and working environment. When you’ve done that, you can then highlight what sort of values and skills your are after. A slight but a huge difference: instead of describing a role, you describe who YOU are as a business or team and who you’re after.

Issues with PDs in a nutshell

To summarise the issues I have with position descriptions:

PDs take a very long time to create.
They are also a snapshot in time and don’t change over time while the role, expectations, skillset and even the industry are constantly changing.
So to keep PDs relevant, you would have to spend a lot of time maintaining them.
That’s a lot of effort for no value because (hopefully) no one uses them.
Well, except for performance reviews I guess.
But performance reviews are such a bad idea to begin with.

An alternative to position description

I don’t think anyone gets off the bed in the morning and thinks: “I am going to be a really crappy employee today, produce a little bit of outcome with shitty quality, and make my team and manager’s life difficult”. Most employees want to be the best. They want to do high quality work. They just don’t know how to go about doing it. They don’t know what they’re expected to do. As a leader, you should be clear and explicit about your values and expectations. Then, if there is misalignment between your values and what your employee does, talk with them straightaway, understand their point of view and help them understand yours. Coach and mentor them, and learn from them. Likewise and as importantly, if they do something great, acknowledge and reward it.

But what if I train them and they leave?

What if you don’t, and they stay?!

Some people are not coachable and are making a negative impact on my team”

Nothing brews as fast as negativity in a team. I have touched on this in an old post titled dealing with net negative producing programmers/personnel. Don’t waste your time trying to coach the uncoachable or shoehorn someone into your company’s culture. If they don’t fit, make it clear, ask them to leave and put yourself, your team and the misfit out of misery. You should also reflect on how you got there. Was it a bad hire? If yes, is there anything you can do to improve your hiring process and avoid this in the future? Or was it mismanagement? Could that person have been managed to be much more productive and effective? Learn from your mistakes, improve and move on.

Be clear and explicit about your values and expectations

Thu, 29 Jun 2017 00:00:00 +0000

I recently joined Simple as the CTO. Inspired by Doug Conant, I communicated my values and expectations with everyone in the team and organisation very clearly, and this has had a big positive impact on my relationship with my colleagues.

The people with whom you work are not mind readers. Tell them who you are, what’s important to you, and how you intend to lead.
— Doug Conant (@DougConant) November 7, 2016

Before I started

Before I started, I sent an email to the CEO and the technology leadership team to discuss what I stand for. After highlighting my pledge and expectations (more about this further down), the email ended with:

I will communicate these with the entire team on my first day at Simple. If you have concerns about any of these please let me know and we can discuss them; otherwise I expect full support from you to deliver on these.

Thanks

The response I got from everyone was quite positive. So I arranged a meeting with all head office staff at 9:30AM on my first day.

First Day

This is the gist of what I discussed with everyone on the first day (omitted the personal introduction for brevity).

I am here on a mission. My mission is to build a world class software team that defies any challenge and creates products that customers love.

To succeed in this mission, I make a pledge to you and have a few expectations from you.

My pledge:

I respect each and everyone of you.
I act with integrity. Every word coming out of my mouth is a promise and a commitment that you can hold me accountable for. If I fail to deliver on my commitment, I will openly acknowledge it.
I provide full transparency around circumstances, plans and decisions as well as organisation’s expectations of you.
I learn about your career goals and aspirations and will help you grow and realise them.
I collaborate with you to create an environment that is conducive of excellent outcomes and I will recognise and reward excellence.

And team leads share this pledge.

Any questions?

My expectations:

You respect everyone.
You act with integrity: you do what you say you do and I will hold you accountable for it.
You provide full transparency: you are accountable for your actions and own and communicate the results.
You have high level of sustainable productivity and quality. This is not a place for mediocrity. To be part of a world class team, you have to show world class performance.
You value collaboration over individual goals. It doesn’t matter how good you are if you are not willing to collaborate. High performing individuals are great. What’s greater is high performing teams and what’s even greater is a high performing company.
We only make new mistakes. We forgive any mistake the first time around, but we also immediately learn about and address the root cause so it won’t happen again.

And team leads share these expectations.

Any questions?

Please think about these in the next few days. It’s ok if you don’t agree with these; but you need to raise and discuss it with me. So if you have any questions, comments, concerns or objections, please come to me and let’s discuss them in person. If I don’t hear anything from you by the end of week, I will take that as a commitment from you.

I then documented this on the company wiki and sent an email to everyone thanking them for their time and sharing the pledge link with them. We held another meeting the week after to discuss any questions or concerns.

Key Takeaways

Think about what you stand for and expect before communicating it to anyone. I thought about my values for over a week and while a lot of things came to mind, I decided to keep the list very short so it’s easy to remember.
Think about the end goal and communicate that as well. I set a mission for myself, to build a world class software team that defies any challenge and creates products that customers love, and I communicated that with everyone, so you are all aligned and headed towards the same end goal.
Get buy-in from the CEO (or your immediate manager) and the rest of the leadership team before communicating this widely. It doesn’t matter what you stand for if you values are not aligned with the CEO’s. Likewise, if you don’t work with everyone directly, it’s critical that other leaders share these values and expectations; otherwise you could fail to deliver on your pledge or engender your expectations because a leader is misaligned.
Communicate your values and expectations early and clearly up, down and sideways.

Last but not least, you can’t be selective about when to deliver on your promises and stand for your expectation. Communicating values you can’t live up to and stand for consistently is much worse than not communicating them at all. I have pledged that “every word coming out of my mouth is a promise and a commitment”, and that applies to my pledge and expectations. This is not always easy and could be quite painful at times; but I strongly believe that nothing works without integrity.

The ultimate software QA process

Thu, 30 Mar 2017 00:00:00 +0000

I have worked with many teams and organisations with varying approaches to testing and QA. In this post I cover some of the approaches I’ve come across and their pros and cons, and will then present my ideal software quality process.

ALL the QAs

On the one extreme side of QA heaviness spectrum, I’ve worked with a few teams with many layers of manual testers. In one team, there were no automated tests whatsoever. When developers were done with a task, the BA would pick up the task and test it on the dev environment. If the BA was happy with it, the task would move forward in the testing pipeline where it was later tested by a QA team. If the QAs were happy with it, it would then move to the next and last stage where yet another group of QAs, called acceptance testers, would test the change again but in a broader context to make sure it hasn’t caused any regression elsewhere. The testing process in this team took somewhere between one to six months depending on the size and criticality of the change as well as the testing pipeline which was always full. If I recall correctly, this team had 47 staff: there were eight developers, six BAs and the rest were testers!!

In another team, there were some automated tests as well as three manual testers per developer and even then the manual testing cycle took as long as development. I was invited to this organisation to help them improve their processes. When I asked why they had so many testers, they said their application is life critical and a bug could cost someone’s life. When I looked at their testing pipeline, there were several critical bugs sitting there waiting to be tested and released. When I asked them why they don’t release the bug fixes, they said they couldn’t release them before they are thoroughly tested and that took weeks. Oh the irony!!

QA as the quality gatekeeper

When you rely very heavily on QAs for the quality of your software, you end up adding more and more QAs as the size and complexity of your app grow, and even then your release cycle gets slower over time.

The other side effect of this process is that developers become more and more insensitive towards software quality because “it’s someone else’s problem”. This then leads into bad blood between QAs and developers: QAs become increasingly unhappy as developers don’t care about quality as much as they should and QAs find a lot of silly bugs on every card. Developers also become increasingly unhappy at work as they keep getting pulled back to fix bugs on tasks they considered done.

You might think “but good devs don’t behave this way”. I have seen many good devs become lazy and insensitive towards quality when someone else is responsible for it.

This buck passing mindset is what brings the application delivery cycle to a halt. When you hold your QAs responsible for the quality of application, they make damn sure that no bug can get through. So they test the application as thoroughly as humanly possible, and when that takes a long time and business wants features out more quickly, you hire more QAs. This is a slippery slope.

No QAs!

On the other extreme end of this spectrum, I’ve worked with teams without any official QA role. In these teams, in absence of anyone else to rely on for software quality, the developers write high quality code with good test coverage. They also manually test their own code with the provided acceptance criteria, and they do some manual testing for edge cases, security, accessibility, UX etc. Typically the change is made, reviewed and tested by different developers to ensure it’s looked at by as many people as possible.

When you don’t rely on QAs for software quality, the application can be released to production more frequently, because there is no back and forth between developers and QAs and there is no separate testing pipeline. In the teams I mentioned above, there were multiple releases to production everyday: basically if the developers were happy with a change and the CI pipeline was happy with the build and tests, the code would be released to production. More often than not, the release is configured to happen automatically at the end of a successful test run. So the speed of value delivery is significantly higher. Another perhaps intangible effect of this process is that developers are really happy at work because they feel ownership over the quality of application, get to deliver value to customers at a rapid pace and aren’t bogged down by “unnecessary red tape”.

Some people get concerned that this results into some bugs getting released to production, and they are partially right, as I discuss below. That said, since you can release more frequently, the bugs can be dealt with very rapidly. A bug is found or reported? You write a failing test to prove the existence of bug, you fix the code, and go through the same process as above, and the bug fix is pushed to production an hour or two after the bug is found.

You still need QAs

Having devs as quality gatekeepers is not all rosy though. When it comes to software quality, QAs are a lot more detailed, thorough and picky than developers. As a QA, you have a different mindset. You look at the app through a different lens. Your job is basically to crash the application or make it misbehave. Developers are good in creating applications, QAs are good in breaking them.

Also why get developers who are good at creating and enjoy doing it do something they’re not very good at and don’t necessarily enjoy doing? Having QAs in your team means better quality outcome. Also since your developers spend less time on manual testing, they will have more time for coding and you can get more features out.

The ultimate software QA process

You should have QAs in your team. Your milage might vary but I think this works very well with one QA per 5 to 8 developers depending on several factors (e.g. how good the devs are, how good the QAs are, how much test coverage you have, the maintainability of your codebase etc). Your QAs are not in a different team. They are PART of the team. Your QAs work closely with BAs to define the story and acceptance criteria before it’s picked by developers. They attend the story kick off with BAs and developers to highlight and discuss corner cases, gotchas and pitfalls. As developers work through the stories, they pair with them to refine the acceptance criteria and help define acceptance tests, help write automated tests, test the feature as it’s being coded, answer questions along with BAs and basically work on the story alongside the rest of the team. This way, by the time the story is done, it should be properly tested as well and there should be no surprises for anyone. So the story is not dev done. It is DONE and it could go to production if it passes the peer review and the build. But the work of QA is not done yet.

Exploratory testing

In absence of test automation, most QAs come up with large body of test cases and run through them manually. QAs should not follow test scripts. Computers are really good and fast at following scripts. Humans are too smart and valuable for that. If your QAs follow a test script, automate it. That will run significantly faster, can even run overnight and since computers don’t get tired or distracted, it is not prone to human error!! This will also free up your QAs to do what humans are good at: creative work.

I have found QAs to be very effective in exploratory testing. That’s when they do their best to break the system; but we just said the code is pushed to production if it passes tests! In this model, there is no place for a test environment, because it unnecessarily slows down deployment to production. That leaves QAs with one place to test the application: in production! After the code is released to production, QAs will test the feature and application in production. You can never ensure the quality of an application better than testing it in production when it’s run on the production hardware, configuration, setup etc. And BTW this applies to performance and load testing too!

Change gradually

This sounds like a scary concept and admittedly takes a lot of discipline and practice to master; but I’ve done it in a few teams and nothing feels as good or effective in comparison. I appreciate it is difficult to go from a traditional QA heavy mindset to what seems like a cowboy style software delivery. There are different things you can do to bridge this gap and build the trust. For example you can use something like GitHub Flow where you deploy branches to production. That way if you find a bug in production, you can just push the master back up again. Obviously there are considerations around database migration if you have relational databases; e.g. your database migrations should be backwards compatible so the old version of application can still work with the performed database changes; otherwise pushing master to production could break the application or cause data loss (if you roll your database back as the result).

Also don’t forgo your test or staging environment instantly. You can use them while you’re building up and getting used to this process. Keep the test environment and run your manual regression testing on them as a stage before pushing the code to production. Every bug you find will obviously reduce your trust for full continuous delivery model promised above. That means you either don’t have good test coverage or good tests, or your devs are not disciplined enough, or your QAs are not properly involved during the development of stories. You need to chip away at these issues gradually. As you get better at it and the number of bugs you find during this stage gets closer to zero, you build up trust in the process and at some point you will be able to comfortably drop the manual testing stage altogether.

You might need a manual test stage for some changes

Some changes are too big, complex or otherwise difficult to make in a backwards compatible way. It’s not that you can’t do it; but it might cost significantly more if you did it that way. You would want to test these changes manually to avoid any critical bugs slipping through or bringing production down on release. This can happen by pushing the code in a feature branch and push that branch to a test environment for manual testing (similar to how GitHub flow works). You can then either merge that back to trunk and push to prod, or just push to production from the branch and only merge to master after a successful release.

Feature Toggles

I see feature toggles as an essential ingredient for continuous delivery. If your changes are wrapped in a feature toggle, then they can be deployed to production with less worry. QAs could then login to production, toggle the feature on for themselves, test the system with the feature toggled on, and only toggle the feature on globally (AKA release it) when they are happy with its quality.

You can also automate the feature toggle roll out. You basically start with the feature completely toggled off. When the QA is happy with the feature, they trigger an event in the system that will automatically roll out the feature to a small test group. After the rollout, the test group’s error count, conversion rate, memory usage patterns, CPU usage patterns etc is automatically compared with a control group. If the outcome is desirable, the system rolls the feature out to more users and continues with the same process until the feature is rolled out to all users. If the test group shows any spikes in the number of errors or any other important indicator, then the system automatically toggles the feature off. In one project, we went as far as identifying the PR that led to run time regression and raised a ticket with data collected from the experiment and assigned it to the team as a high priority. This is a very interesting approach; but I’m going to park this here as it’s not the focus of this article.

The ultimate software delivery process

Let’s zoom out and look at the entire software delivery process. The ultimate software delivery process is the one that allows for rapid and continuous development and delivery of high quality features to customers.

This idea of QAs working along the team is not a new concept. It is just an extension of the DevOps movement. DevOps was created out of the frustration caused by the functional silos between dev and ops teams. Dev would throw their application over the wall to Ops team for them to deploy, monitor and operate it in production. That led to miscommunication, issues in production, slow releases, politics and heavy change management processes to name a few. Devs throwing the application over the wall to QA team is no different. You still end up with functional silos which result in more or less the same amount of pain, slowness and politics.

Now let’s expand this idea to every role in your team. If BAs analyse the requirements, fully specify what needs to happen and throw it over the wall to devs, then devs become code monkeys and just implement what’s written on the card. Instead you want discussion between BAs and devs. Devs should discuss the stories in detail with BAs, challenge things that may not work well and work with BAs to define the work in a way that makes sense, not just from the business’ point of view but also from the technical one. Likewise, you don’t want your UX designers to design interfaces, workflows and interactions without consultation with the rest of the team or your Product Manager(s) to come up with a roadmap in isolation.

Everyone is in this together and this is why cross functional teams work so well. The smoothest software development process is one achieved through heavy collaboration on every aspect of work, from analysis to design to development to testing and deployment. The days of waterfall software development where these tasks were performed sequentially and by different groups of people are long gone. A software development team should now be a self contained team composed of every role required to go from an idea to a running feature in production.

I would love to hear your thoughts if you have had success with other processes and approaches.

Microservices lessons from trenches - DDD Melbourne

Fri, 07 Aug 2015 00:00:00 +0000

You can find the screen recording and the slides from my DDD Melbourne talk titled ‘Microservices lessons from trenches’ here:

Microservices lessons from trenches from Mehdi Khalili

Microservices lessons from trenches

Wed, 05 Aug 2015 00:00:00 +0000

There has been a bit of hype lately around microservices. You go to conferences and read blogs and all you hear is how awesome microservices architecture is. Some of the main (perceived) benefits of microservices are:

Scalability: unlike monolithic apps, you can scale the services that actually need to scale instead of the whole app.
Loose coupling: loosely coupled services can be versioned (or rewritten from the ground up) independently without impacting the rest of the system.
Fault tolerance: due to loose coupling, isolated processes and hosting, faults in one service doesn’t impact the rest of the system.
Scalable development: you can’t throw 50 developers on one codebase as that would lead to a lot of merge conflicts; but you could throw 50 developers at 10 repositories without any problems.
Lower cognitive load: instead of having 1 million lines of entangled code in one repository, each service has its own repository which is a lot lighter. That means you can wrap your head around a service end to end a lot simpler.
Technical diversification: since you have different codebase and data storage for each service, you can use the best tool for the job in each service: use relational DB for one service and graph DB for another, use Node for one service and Java for another etc.
Autonomous teams: each service typically has a service team that owns it, from perception and design all the way to implementation, deployment and monitoring. Ideally a service team has complete autonomy over the service: they get to pick the technology they want to use and implement the way it makes sense to them, as opposed to having to follow a single coding standard and tech stack which is typical of monolithic applications.
Continuous delivery: it’s a lot simpler to do continuous delivery of smaller independent services and components than to continuously release an entire application all at once. Loose coupling and fault tolerance also make it easier to deploy frequently without impacting the entire system.
Evolutionary design: smaller services are easier to create, change and evolve and that facilitates experimentation. Create a service to validate an idea. If it flies, keep it and evolve it, and if not, kill it. To do this effectively you will need deployment and infrastructure automation.

I purposely didn’t go into details of these benefits because they’ve been written and talked about many times. If you’re interested you can learn more about microservices and their benefits and challenges here.

I love microservices and distributed systems and that’s my main expertise - what I don’t like is the hype. Developers are very passionate about what they do. We hear or see something awesome and suddenly HAVE to do it, not because it fits our needs but because we like to try it. So in this series of posts I would like to talk more about some of the challenges of microservices.

Here is an (evolving) index of posts I am considering to write over time:

Culture
Fault tolerance
Loose coupling
Greenfield microservices
Service size and boundaries
Continuous delivery
Decentralized data management
REST or messaging?

I hope that by highlighting some of these challenges, you weigh the challenges and cost of microservices against its benefits before you pick it for your next project.

Stay tuned!

BDDfy V4

Sun, 27 Jul 2014 00:00:00 +0000

After 4 months of hard work, 30 pull requests and 300 commits BDDfy V4 is now released with a lot of cool new features. The TestStack team is super excited about this release and we hope you will be excited to see the new features too.

Examples

BDDfy now supports Cucumber-like examples on both Fluent and Reflective APIs.

This is how you can use examples in the fluent API (all of the samples are using NUnit and are available from the TestStack.BDDfy.Samples NuGet package):

public class RunExamplesWithReflectiveApi
{
    public int Start { get; set; }
    public int Eat { get; set; }
    public int Left { get; set; }

    [Test]
    public void CanRunExamplesWithReflectiveApi()
    {
        this.Given("Given I have <start> cucumbers")
                .And(_ => _.AndIStealTwoMore())
            .When(_ => _.WhenIEat__eat__Cucumbers())
            .Then(_ => _.ThenIShouldHave__left__Cucumbers())
            .WithExamples(new ExampleTable("Start", "Eat", "Left")
            {
                {12, 5, 9},
                {20, 5, 17}
            })
            .BDDfy();
    }

    // I didn't have to create this method
    // because all it was going to do was to set Start property
    // which is being handled by the framework
    // And the step title is provided inline
    //private void GivenIHave__start__Cucumbers()
    //{
    //}

    private void AndIStealTwoMore()
    {
        Start += 2;
    }

    private void WhenIEat__eat__Cucumbers()
    {
        // This method is called after the Eat property is set by the framework
        // I didn't have to put this here, like the Given method, but I put it here to show that
        // you can take additional actions here if you want
    }

    private void ThenIShouldHave__left__Cucumbers()
    {
        Assert.That(Start - Eat, Is.EqualTo(Left));
    }
}

Here is the generated console report:

and the HTML report:

You could also write examples in the reflective API:

public class UseExamplesWithReflectiveApi
{
    private int _start;
    private int _eat;

    [Test]
    public void CanRunExamplesWithReflectiveApi()
    {
        this.WithExamples(new ExampleTable("Start", "Eat", "Left")
            {
                {12, 5, 7},
                {20, 5, 15}
            })
            .BDDfy();
    }

    void GivenThereAre__start__Cucumbers(int start)
    {
        // the start argument is provided by the framework based on the example entries
        // please note that `start` argument name matches the `start` header from the examples
        // and also it has to match with the <start> placeholder in the step title which is created based on conventions
        _start = start;
    }

    // I can still override the step type using the `Executable` attributes
    [AndGiven("And I eat <eat> of them")]
    void WhenIEatAFewCucumbers(int eat)
    {
        // the eat argument is provided by the framework based on the example entries
        // please note that `eat` argument name matches the `start` header from the examples
        // and also it has to match the <eat> placeholder in the step title
        _eat = eat;
    }

    void ThenIShouldHave__left__Cucumbers(int left)
    {
        // like given and when steps left is provided here because it matches the example header and is found on the step title
        Assert.That(_start - _eat, Is.EqualTo(left));
    }
}

For more information on the example support refer to this post by Jake Ginnivan. We would not have examples support without his huge contribution.

Improvements on Story

Before V4 Story was relatively rigid. We had one Story attribute with As a, I want, so that syntax and it would show up with Story prefix in the reports. These limitations have now been lifted.

StoryNarrative

Story is now a subclass of StoryNarrative class:

public class StoryNarrativeAttribute : Attribute
{
    public string Title { get; set; }
    public string TitlePrefix { get; set; }
    public string Narrative1 { get; set; }
    public string Narrative2 { get; set; }
    public string Narrative3 { get; set; }

    // rest of class removed for brevity
}

There are a few things of note:

There is a TitlePrefix property that allows you to specify what prefix should be used for “stories” in the reports. For example if you prefer the word Specification over Story you can create a new SpecificationAttribute that subclasses StoryNarrative attribute where TitlePrefix is set to Specification (or you could provide your desired prefix for each story if you wanted to).
StoryNarrative doesn’t make any assumptions around story narrative format. It only knows there could be up to three lines in the story narrative. So you could create your custom attribute to tell the story in a different way if you want; e.g.

public class InOrderToStoryAttribute : StoryNarrativeAttribute
{
	public string InOrderTo
	{
	    get { return Narrative1; }
	    set { Narrative1 = value; }
	}

	public string AsA
	{
	    get { return Narrative2; }
	    set { Narrative2 = value; }
	}

	public string IWant
	{
	    get { return Narrative3; }
	    set { Narrative3 = value; }
	}
}

And use it instead of the built-in Story attribute:

[InOrderToStory(
    InOrderTo = "In order to do BDD properly",
    AsA = "As a great programmer",
    IWant = "I want to use BDDfy")]
public class GreatProgrammersUseBddfy

This attribute is not part of BDDfy core at the moment. In order to syntax has gained a fair bit of traction in the BDD community and we would like to have it in the core but the name, InOrderToStoryAttribute, is a bit awkward. We could alternatively add the InOrderTo property to the Story attribute. We need your feedback on this.

Also StoryMetadata class now only relies on narrative lines; so you can completely skip over StoryNarrative attribute and inject the StoryMetadata directly based on some completely different means. You might find more information around this extension point here.

Narrative convention

Starting V4 you can leave out the As a, I want and So that from your story narratives. So instead of:

[Story(
    AsA = "As a great programmer",
    IWant = "I want to use BDDfy",
    SoThat =  "So that I can do BDD properly")]

you can now use

[Story(
    AsA = "great programmer",
    IWant = "to use BDDfy",
    SoThat =  "I can do BDD properly")]

And BDDfy will inject the missing text for you. This is backward compatible and only injects the text if it’s missing.

We have applied the same logic to the Fluent API step titles (more on this shortly).

Complex flows in Fluent API

A couple of years ago I thought complex flows are anti-pattern in BDD and UI testing which is why you couldn’t write a when or given after a then. Well, I was wrong. Now I believe complex flow is a critical part of Acceptance Test Driven Development. So we’ve added support for complex flows in Fluent API. Here is an example:

this.Given(x => x.IAmOnHomePage())
	.When(x => x.ILogin())
	.Then(x => x.IShouldBeOnTheHomePage())
		.And(x => x.IShouldBeAuthenticated())
	.When(x => x.ILogOut())
	.Then(x => x.IShouldReturnToHomePage())
		.And(x => x.IShouldNoLongerBeAuthenticated())
	.BDDfy();

The Fluent API has completely opened up to allow for any combination of steps. This feature is currently missing from the reflective API mostly because applying the order in the reflective API is a bit more complex as it only relies on reflection. Don’t worry though. We have a few ideas for adding that to reflective API and hopefully that’ll surface in the near future.

More goodness on Fluent API

As mentioned above, step type is now prepended to step title if you don’t specify it. For example .Given(() => Foo()) will report as Given foo. This allows for easy code reuse where for example you can use Foo as Given and When. This change is backward compatible so your .Given(() => GivenFoo()) will still work as before.
Fluent API now also supports inline steps which means you can just use an anonymous function inline with the step definition; e.g. .Given(() => { // some code in here // }, "Some title"). Since BDDfy can’t resolve the step title from the anonymous function, you have to pass the title in as the second argument.
We also support title-only steps in Fluent API; e.g. .When(() => "something happens"). This is quite handy in situations where you don’t want to execute a method but just want to communicate something on the report.
We now also support deep method calls on Fluent API so you could write .Given(() => someClass.SomeOtherClass.SomeMethod()). Another cool little feature in Fluent API is the introduction of StepTitle attribute. You could decorate your step methods with StepTitle and just call the method from the step definition API and let BDDfy extract the title from the attribute. This combined with deep method calls allows you to delegate the step title generation to methods on classes defined out of your scenarios. If you are for example using page objects for UI automation, you could put the step title on your page actions! This violates SRP but nevertheless is very cool and it cuts down on the amount of code you have to write in your scenarios.
Step arguments are now evaluated lazily in Fluent API so if previous steps cause the value to change, the value at the time of step execution will be reported in the step.
Step arguments can now also be method calls. BDDfy will try to execute the method and inject the return value as the step argument; e.g. .Given(() => SomethingIsSet(SomeArgument(), AnotherArgument())). We also added support for passing local variables as arguments to steps.

Metro-stype HTML report

There is now a new metro-style HTML report for metro lovers. This is not active by default and if you want to use it you have to tell BDDfy: Configurator.BatchProcessors.HtmlMetroReport.Enable();

Michael has written an extensive post about this here.

Self-contained HTML reports

Both HTML reports are now self contained (this was actually shipped in 3.19). This means you will have one file, instead of 3+ files that you had before, that you can easily share from your artifact repository on your CI server, DropBox or just email around. Your custom JavaScript and CSS files are also embedded in the report to ensure there will be one and only one file in the output.

“But what about jQuery?”, you ask! We didn’t want to embed jQuery into the report but at the same time wanted to have a self contained HTML report. So we are resolving jQuery from jQuery CDN. We have also added a ResolveJqueryFromCdn config point, set to true by default. This allows you to embed jQuery into your report if you need to view the reports without an internet connection.

Also worth noting that since everything is being embedded into the HTML report we decided to minify the bddfy.css and bddfy.js files to make the HTML report smaller and the HTML source more readable.

But steps

For full compatibility with Gherkin we now support But steps. But steps are more or less like And steps except that they read as, ermmmm, but!

There is a new ButAttribute for those who love ExecutableAttributes, a naming convention for methods starting with But (ButGiven, ButWhen, ButThen and But), and step method overloads in the Fluent API.

Breaking Changes

It wouldn’t be a major edition without some breaking changes and we have a few; but all for very good reasons. So here we go:

Some long namespaces were removed from the framework so the API becomes more discoverable. This is a rather big breaking change particularly for fluent API users; but also relatively easy to fix. You just need to delete the now-removed namespaces from your using statements. Most of the API is now located directly on the TestStack.BDDfy root namespace.
The Reporters namespaces that you would use to configure BDDfy’s reports has been moved to the root namespace too.
We have renamed a number of types to more accurately reflect their role and usage:
renamed ExecutionStep to Step & Step’s StepTitle to Title
renamed StepExecutionResult to Result as it was used for Scenario and Story too.
renamed StepAction in Step to Action
renamed MetaData to Metadata everywhere
.Net 3.5 support has been discontinuted in V4. So if you want to feel all the love that’s coming to V4 you will need to upgrade to .Net 4+.
Strong naming has been removed. Please don’t leave me any comments around your need for strong naming. Stop strong naming your assemblies. Who would strong name a test assembly (apart from the organization for whom I created BDDfy in the beginning)?! It is a bad idea.
Story.Category has been removed in favour of scenario Tags. This shouldn’t impact anyone as Category was never fully implemented.

Update to V4

V4 has a lot of goodness in it and we are very excited about it. This is the best thing that’s happened to BDD in .Net since BDDfy itself :) Go and update your BDDfy package to V4 and enjoy the new features.

Microsoft MVP Award

Sun, 20 Jul 2014 00:00:00 +0000

I am proud to announce that a couple of weeks ago I received my first Microsoft MVP Award! This award is based on my open source contribution. So I would like to thank those who nominated me and my open source work, folks at Microsoft who pushed for OSS nomination, and last but not least the great community behind my open source projects.

Open source is a rather thankless job. A lot of effort is put into creating, maintaining and promoting an open source project. If you use an OSS project, consider contributing something back. It could be documentation, logo, automation scripts, bug fixes or new features. While you’re at it, you could also nominate the authors for MVP Award. It is a nice way to thank them for their efforts.

Once again, thanks all! This is quite exciting for me :)

Mehdi Khalili

Product launch checklist

User Experience

Product Management

Marketing

Business Processes

Security

Technical

Reinventing myself

Mission Statement

Principles

Personal development practices

Listen, Challenge, Commit!

Listen

Challenge

Commit

Teams with no ownership

Listen, Challenge, Commit

Position descriptions are harmful

How can I do performance reviews without a position description?

But how can I do pay reviews if I am not doing performance reviews?

Without a position description, I don’t know what my manager expects from me!

We have position descriptions and our employees are very passionate, committed and creative

How can I hire without a position description?

Issues with PDs in a nutshell

An alternative to position description

But what if I train them and they leave?

Some people are not coachable and are making a negative impact on my team”

Be clear and explicit about your values and expectations

Before I started

First Day

Key Takeaways

The ultimate software QA process

ALL the QAs

QA as the quality gatekeeper

No QAs!

You still need QAs

The ultimate software QA process

Exploratory testing

Change gradually

You might need a manual test stage for some changes

Feature Toggles

The ultimate software delivery process

Microservices lessons from trenches - DDD Melbourne

Microservices lessons from trenches

BDDfy V4

Examples

Improvements on Story

StoryNarrative

Narrative convention

Complex flows in Fluent API

More goodness on Fluent API

Metro-stype HTML report

Self-contained HTML reports

Tags

But steps

Breaking Changes

Update to V4

Microsoft MVP Award