After years of getting rejected, the talk I submitted to Agile Testing Days in 2017 finally got me accepted. It was such a privilege and an honor to meet so many of the people I'd only know from the internet. Reading through my notes now, much of what I wrote down has become engrained into how I go about my work everyday. What a blessing it was to encounter the people I needed to learn from in my career at the time I needed to learn from them. I'm glad to see how routine their wisdom has become for me.

• Lisi Hocke gave a talk about growth. I've since seen more of her talks and prepared a workshop with her; I see how she embodies this in her approach to the world.
• Gitte Klitgaard reminded me that believing in people will allow them to be better. Stay curious about why people are doing what they do before placing judgment.
• Kim Knup spoke about a zero bug policy (not any bugs in your backlog). I was not ready to hear this message at the time, having come from places with years-old products and tens or hundreds of bugs in the backlog. But now I see exactly what she was describing: the psychological relief that comes from less time in JIRA and fewer meetings about priorities.
• Katrina Clokie pulled up Noah Sussman's reimagining of the testing pyrmaid (lots of small tests, fewer large ones) upside-down as a bug filter. And suddenly, it clicked for me.
• Alex Schladebeck and Huib Schoots challenged me to think on a meta-level about the testing I was doing, to name the skills and techniques I was using. It would help me as I spent the following years sharing exploratory testing skills with other testers.
• Emily Webber spoke about building trust on teams. I find myself recommending the team manual she developed to somebody about once a month, plus she helped spark an idea for a future conference talk I developed.
• Liz Keogh gave me the words I needed to build a safe-to-fail environment for my team members, where failure is an expected, inevitable part of complex systems.

I realized in compiling this blog post that I'd already written about Agile Testing Days just one month after I'd attended in 2017. At that point, I was looking specifically at regression testing advice, which is what I was in the thick of at work. What am I in the thick of now, and when will it become clear to me?

I was on a panel about critical thinking for the Ministry of Testing last week. One of my fellow panelists and commendable ranter Maaike Brinkoff brought up ad hominem (personal) attacks as one example of a failure of critical thinking. It's one of many logical fallacies that are worth exploring further.

Equipping yourself with the name for a thing helps you recognize it when it appears. (Lara Hogan wrote recently about applying the skill, of being able to name the problem in the room, to defuse tense meetings.) These are some of the fallacies I've across recently when I've been debriefing testing sessions, facilitating refinement sessions, and reviewing conference submissions.

Affirming the consequent

Affirming the consequent is applying a conditional without the conditionality, or assuming something happened because you see a result.

1. If P (I run the pipeline) then Q (the latest build will be available on the test environment)
2. Q (the latest build is available on the test environment)
3. Therefore P (I ran the pipeline)

We can't assume the converse: if Q, then P. Just because the latest build is on the test environment doesn't mean I ran the pipeline. Maybe someone else ran the pipeline, or put the build there manually. Maybe there haven't been any changes since yesterday, and the build from yesterday is still the latest one.

Fallacy of composition

This assumes that something that applies to one member of a class applies to them all.

1. Y is part of X (Stephanie is an admin user)
2. Y has property P (Stephanie can see this page)
3. X has property P (any admin user can see this page)

We can't assume that what's true for one member of a class applies to all of them. What happens if Stephanie can be assigned more than one role, a more restrictive/regular user role in addition to the admin role? Can she still see it? What if Stephanie being able to see the page has nothing to do with her status as an admin user?

Post hoc ergo propter hoc (correlation without causation)

This one is easiest to see when others are debreifing their testing to me, but I've also learned to catch for myself.

1. Event A occurred (I clicked the button)
2. Then event B occurred (a whole bunch of log messages appeared)
3. Therefore A caused B (clicking the button caused a whole bunch of log messages)

We can't assume that events that occur in a particular sequence in time are necessarily causal. Did clicking the button trigger the log messages? What do the log messages say? Did you read them? Who else could be using this environment? Does the same thing happen every time you click the button, or when you run the application in a different environment?

Argument from repetition

When someone says the same thing enough times, or brings up the same unimportant issue in a refinement meeting week after week, it can become easier to address the issue rather than convincing them yet again why it's not a priority. I've been facilitating refinement meetings every week for my teams for the past two years. I only have a finite amount of energy that is not always worth expending by refuting the case for a small edge case week after week.

Shoutout to my logic professor Dan Cohen at Colby College, who had us memorize and distinguish logical fallacies as part of his brilliant Logic and Argumentation course, and pointing out that an ease and comfort with truth tables would translate well to a computer science course. Special thanks to Joep Schuurkes for his philosophical and technological opinions on this piece.

TestCraftCamp, the unconference I co-organize, had its third installment yesterday. I'm glad I had enough energy to attend more sessions this time, though of course the "too many good things happen at once" problem remained, as it does with any valuable conference. I was able to join a discussion Maaret Pyhäjärvi (who is so often a driving force behind gatherings like these) about low and high value work, a discussion Joep Schuurkes led on daily writing practices, part of a session Veerle Verhagen hosted on testing without touching (before I realized I'd spent too long looking at the product under test already), as well as finding some of my brethren in between the sessions.

I took notes in a sprawling mind map for another session Veerle hosted, which she pitched as "spicing up your long-term relationship! (with your project)." Besides the memorable pitch, I was excited to see this topic on the schedule for a couple reasons:

First, I've been working on my current product for two years. Some new people have joined the project recently, and having to explain and document the intricacies of our product keeps giving me new reasons to question the decisions we've made as the product has been developed.

Second, I don't think this it's something we typically acknowledge: our skills can get stale and our motivation can wane when we've developed a certain level of comfort and mastery with a particular team and product. Even without changing jobs or teams, it can be worthwhile to shake things up.

On my project, adding permanent designers and engineers to the team is shaking things up. The hive mind in the session came up with many other ways to get new people involved in testing, if temporarily:

• hold a bug hunt or an ensemble
• teach an intern
• swap products with another tester
• bring your product to a meetup
• pay for crowdsourced testing

I collected a sprawling list of people's ideas for how you as an individual can gain perspective and change the way you're looking at your work. Taking notes on a multi-faceted conversation is hard, and in reviewing them, I realize they fall better into these groups:

• identify the assumptions you've built over time (which aren't true? which can be discarded? what's hard to talk about? how can I change perspective?)
• think about risks (nightmare headlines, investigate competitors, talk to users, riskstorming)
• use different testing personas (extreme conditions, soap operas, testing tours, dogfooding/drinking your own champagne, this)
• step back (take a break, go on vacation, switch projects)

As a counterpoint, the group in the session did acknowledge that reaching a point of comfort and mastery can be a good thing. (See chapter 4 of Jerry Weinberg's Becoming a Technical Leader for more on plateaus and ravines.) After months or years on a project, you'll know where to look first for what your developers typically miss. You'll know what's not worth testing. In Veerle's case, the application has been getting 5-star reviews, which provides one positive angle on product quality.

Thanks so much to the facilitators, participants, and other organizers of TestCraftCamp. It's so fulfilling to get to the end of a long day and hear that we created a "safe environment," people who were only planning to stay for the morning lasted all day, and people were able to take breaks when they needed to. I'm glad we were able to fill others back up with energy.

In 2015 when TestBash came to the United States for the first time, it was to New York. I was living in the city, but I was stuck at a job that wouldn't pay for a ticket. "Ask Rosie if you can volunteer," my mentor/sponsor/benefactor Martin Hynie suggested. Rosie, who I knew only as the lady who'd mailed me Ministry of Testing stickers from England for...no reason at all, obviously let me in.

People ask "can we do it?" instead of asking "should we do it?"

~Keith Klain

I got to crash the speakers dinner. I got to go to Selena Delesie's workshop about leadership and change. At the end of the day, she praised my active participation and thanked me for being in her workshop. I was already confident in my testing skills, but she helped me see myself as a potential leader. In a follow-up coaching session I had with Selena about negotiating a higher salary, she asked me why I wanted more money. I'd recently moved into an apartment by myself, and couldn't think of what I would do with more money. It's something I think about with every job change, every growth in title and responsibility. Realizing I didn't want or need more money was a crucial step on the path to life-changing relocation.

The common denominator in all your dysfunctional relationships is you.

~Keith Klain

During a break between talks the following day, I snuck on stage wearing "the" Ministry of Testing tutu. During another, I wrote a bunch of notes to give a 99-second talk about leaving a closing comment on a story (which I completely forgot about, before later writing on this topic for the Dojo). While waiting on stage behind dozens of people waiting to give their 99-second talks, I improvised one about moonwalking instead.

The 99-second talk I didn't give

I met Helena Jeret-Mäe, Maaret Pyhäjärvi, Dan Ashby, who along with the conference friendships I was just beginning to foster, gave me a vision of what my career could be, and where it could go at a time when I knew I needed something different. Helena saw my talk at Let's Test the following spring, and gave me valuable critical feedback that helped shaped future talks. Maaret introduced me to strong-style pairing, which changed the way I worked with my colleagues to this day. Dan had me on his podcast, reinforcing for me the success of my talk at the following year's TestBash USA.

Write down when you receive a compliment. Maybe it's true.

~Helena Jeret-Mäe

Why am I revisiting my notes from this conference six years later? I might be feeling a bit nostalgic for the seeing-people-in-person events a year into pandemic-induced isolation. I'm also in the middle of reading "Becoming a Technical Leader" by Jerry Weinberg. One of the questions asks you to read an autobiography of someone you admire. It turns out none of the people I shared TestBash NYC with have published autobiographies...yet.

Check that you have time

At my current job, everyone in the R&D department gets a budget of two days per month to spend "crafting," or researching, building, testing, etc. what interests them. Most often people use this time to bring work they're passionate about up from the bottom of the backlog to be worked on now.

I was in the middle of adding a security scanning tool to our CI pipeline when I happened upon something interesting to test: a new web application. An old application that had been widely-used around the company, and as it turns out, with our customers too, had gone down. It allowed you to share a piece of text with a link. The text was only visible once. It wouldn't be viewable subsequent times you followed the link, so the application was good for sending passwords around securely.

The person who'd built the application had left the company. In inquiring about who/how we might maintain it now, one of the customer support leads mentioned in a public Slack channel that he'd built a replacement. "Great!" I thought. "This is the one day I have an hour to test it." I was waiting for code review feedback on my pipeline scan, so it was perfect timing.

Check that the feedback will be heard

It's a waste of time and energy (with the latter being in shorter supply these days) to test something if nobody's going to do anything with the results of your testing. More on that in this post. So I checked with the customer support person first. After taking a quick look at the new application and confirming that it seemed to work, but could use some tweaks, I asked the customer support person in a direct message if he was ready for usability and accessibility feedback.

Me: Hi there, I have a bit of usability and accessibility feedback about the secrets app. Is it at a stage where this feedback would be useful?

Him: Yes, definitely 🙂

Me: Great, I'll send you a Paper doc this afternoon.

Collect the feedback in the same way it will be presented, and present feedback in a way that the audience is comfortable with

As much as I love making mindmaps, I decided that might not be the best format in this case. I doubted the customer support lead would bother to download a mind mapping application, company security restrictions prevented me from sharing a web-based one, and I'd probably want to walk him through a mind map I produced. But that felt like too much trouble for something small, plus I didn't want another Zoom call on a day otherwise free from them.

Instead, I used my company's go-to document tool of choice: Dropbox Paper. I might not enjoy it as much, but I knew it was a way he was used to receiving and collaborating asynchronously. I confirmed that format with him to be sure, and then I got to testing.

Share your oracles (reasons why you have feedback)

Once I opened a one-time link the application created, there was a page with an animated GIF, the piece of text that was shared, and a button to Copy Value. My immediate testing notes were something like:

• Remove GIF/make it stop rotating
• Move button to the top
• Make font bigger

This customer support lead might have just taken than feedback and made the changes. But since I don't have an existing relationship where I provide feedback about his work, and developing applications is not his normal line of work, I provided more details:

• Animated GIFs (that can't be turned off) can trigger people with epilepsy or motion disorders (vertigo for example). Here's the W3C guideline on this.
• Move the Copy Value button above the text you're sharing so it's visible even if the text is ~5000 characters long.
• Increase the font size from the current 14px to 16px for vision-impaired peoeple. The ADA and typography geeks recommend 16, Apple has 17 as their font size.

Now the customer support lead understands why I'm asking for these changes. He can make different decisions than exactly what I've suggested while still addressing the problem I'm reporting. Plus he'll know more for next time he's building something.

Acknowledge the limits of the situation

Somewhat to my surprise, the customer support lead started implementing my feedback right away! He took what I said into account, including removing the animated GIF entirely. He appreciated my feedback and wanted me to look at the application again the next day. Unfortunately, the next day was back to a regular work day filled with priorities, pressure, meetings, etc. I told him I wasn't going to have time to test it again. He went ahead and launched a functioning product.

Move on

While testing the product, another colleague noticed that the application had a larger architecture and setup than strictly what was needed for this particular use-case. Also, the application didn't provide an API for applications instead of humans to be sending around secret links to other humans. I defended the customer support lead: he was doing his best to solve his problem with the tools and skills he had. It wasn't the right time or place to come in at the end of a project that was about to provide value to people (myself included) and announce "this was not the optimal way to build this tool." You don't have to share all the feedback you collect.

Summary

The next time you're wondering if you should parachute in to test something new, consider these steps:

• check that you have time
• check that the feedback will be heard
• collect the feedback in the same way it will be presented
• present feedback in a way that the audience is comfortable with
• share your oracles (reasons why you have feedback)
• acknowledge the limits of the situation
• move on

I answered a bunch of questions for a Ministry of Testing Ask Me Anything session on Test Reporting this week. You can witness me filling an hour here, or read my answers to the questions we didn't have time for here. Here's a summary of my strongest opinions:

1. A test report is not one particular thing.

A test report may be written words. It may be a chart. It may be delivered verbally. But I don't have a template, or things I usually include, because:

2. A test report should be tailored to your audience.

Who should be listening? What do they care about? If you're reporting about your testing to a developer on your team and the CEO of your company, you'll need two different kinds of reports.

3. A test report is the beginning of a conversation, not the end.

Presenting a test report allows you to gather information on what matters to your stakeholders, involve your team to streamline your process, and reflect on the quality of your testing for yourself. Feed that back into what and how you're going to test next.

Resources I'd recommend for more on how to report about your testing:

• Lisa Crispin on Agile test reporting
• James Bach on professional test reports
• The Four-Hour Tester on bug reporting

I was about to throw away a whole notebook full of conference notes I'd already blogged about. But because I'd already shipped these notes across an ocean and ignored them for a few years, I figured I'd give them a final read first. Luckily I found a few pages of gems.

Karen Johnson's talk at CAST (The Conference for the Association of Software Testing) in 2015 covered the breath and depth of her experience in the software industry. I've taken on the career coaching role for some colleagues, and so much of what Karen emphasized is what I've been saying to them as they try to advocate for that promotion, discover their next step, or accept that they have done the best they could given the circumstances.

Testers should have a portfolio

How do you make your work visible? Particularly when your work is invisible when done well? Tell people about it in standups. Tell your boss at your 1-on-1. Write it down and then share a link the next time it comes up.

If you were to do a retro on yourself, what would you want it to be?

Karen advocated for giving back to the community, and challenged the gentleman who pointed out this was not strictly part of the job description. "That's just who I am." The senior roles at my current company are often differentiated not by technical prowess, but by building others up. Sharing to a big audience trips people up, but the steady radiating of information in writing, pairing with people to solve problems, and being the go-to person for questions serves to enrich and scale a company more effectively.

Where's the gap between what you're doing now and what you want to do?

This is something I'm able to concretely address within my company, when I know how the departments are structured and how the job descriptions fit into the org chart. I've made a list (or tasked my colleagues to do this for themselves) comparing their current job description to the one they're aiming for. Next to each item, list what you're already doing to fill that gap, or at least one thing you could be doing in that direction. Sometimes it's a matter of recognizing for yourself how much you already do.

Your boss may never be your mentor

These talks from Marianne Duijst and Martin Hynie dive into this deeper, but find help where you can get it.

In five years you might specialize in something that doesn't exist today

I feel a bit silly asking people in 2021 where they see themselves in the future, or even what they want for themselves. Don't feel bad if you don't have it all planned out.

How to get promoted

Be reliable, dependable, the one to be counted on, and willing to take on more. Who wouldn't want someone like that around all the time?

Karen says during the talk that she doesn't know what mark she will leave on the testing community. I certainly appreciate this bit of her wisdom.

The Exploratory Testing Peer Conference #ET19 followed the 2019 Valencia edition of the European Testing Conference. I said something that I pretty immediately forgot in the upheaval when I returned to the office, but it really stuck with Anne-Marie Charrett. I appreciate both that she remembered this at all, and continues to insist that I'm the one who had this stroke of genius when she really brought it to life. Here's the idea:

It's straightforward to follow one thread or idea through an exploratory testing session. It's not straightforward to decide which path to take, feed information from one path back into another, recognize that there are different paths, or bring others along for this journey.

The Ensemble at Work

We have a weekly ensemble testing session at my workplace. For an hour and a half, testers from different teams working in the same tech stack come together to share knowledge and build testing skills. In a recent ensemble testing session, a tester on one team brought a ticket they'd been avoiding tackling on their own. They knew they didn't know how to test the fix. But they felt like they'd talked about it enough as a team that they should have understood what to do already.

We read through the story with the group of testers. We determined that a static code analysis security scan had discovered vulnerabilities in a couple of libraries. The developers had fixed the issue by removing the libraries. It was our mission to make sure those libraries were removed.

Immediately a plan came to my mind: 1. Map out what kinds of pages there were, assuming that different pages of the same type would be likely to load the same libraries: list view, detail view, landing page, etc. 1. Look at one of each of those pages with the Network tab open in the developer tools.

In a quick spirt of excitement, I dumped this idea on the group without figuring out if people knew what either of these things meant. (It turns out, not everyone did.) But everyone seemed to understand that there was somewhere in the developer tools where we could tell which libraries were loaded, so we started there. Exploring in a group is not about getting everyone to follow my idea immediately (or ever), it's about making sure everyone is on board and understands what's going on.

Proving the absence of a thing is harder than proving the presence of something, so we spent a bit of time looking through the Console and Storage tabs, as well as reloading the page with the Network tab opened, to figure out what appeared where. That helped everyone remember or discover that we didn't need to reload the page if the Network tab was open before the page loaded. This sped us up for the rest of the session.

Next, we looked at a couple of similar-looking list pages. We searched for the libraries in the Network tab. They weren't there. Now that we'd seen a couple of examples, I decided it was the right time to bring up my original idea of grouping pages by type. (Going from the abstract to the concrete doesn't work for everybody, so sometimes going from the concrete to the abstract works better.) I asked "These last two pages both looked like list pages, what other kinds of pages are there? Can we list them? Should we look at a detail page?" This comment blew the mind of the tester who brought this ticket. They'd been testing this product for two years and had never organized the product this way in their brain. It may not have occurred to them that a product could be organized in different ways in their thoughts depending on the circumstance. We got as far as listing the concrete pages we'd checked, but not as far as identifying all the types in the abstract before the energy in the ensemble moved on.

We looked at a detail page. We looked at a settings page. Then one of the testers who's been looking at a lot of front-end Javascript noticed two things: both the URLs we were searching for had ajax in them, so we only needed to search for one thing on each page we opened. And second, they knew that ajax was used to make changes to pages that had already loaded, so they asked "what kinds of pages change after they're loaded?" In this particular application, it was mostly forms in pop-up windows, so we concentrated our efforts there for the rest of the session.

The whole session took about an hour and a half. A tester that came in scared and confused left empowered, with information to bring to their developers, and a plan for how to execute the rest of the testing. Here's one way of looking at our exploratory testing session:

At every stage, we absorbed a lesson as a group, and used it as our new superpower to make our testing better for the next bit. There were other paths we could have pursued, but many of these weren't consciously mentioned or acknowledged during the session.

The Ensemble at the Conference

I facilitated a couple of ensemble sessions with groups at Agile Testing Days. Our first emsemble had a couple people drop out, so it ended up being one tester, one developer, and me. We were looking at a very straightforward application from Alan Richardson where you can decide whether a string contains 7 characters and is valid (in the set A-Z, a-z, 0-9, and * ). A few different times the developer and I asked if we should look at the source code. Rather than trying to interrogate the application based on the behavior from different inputs (black box testing), we wanted to go to the source (white box testing).

But we never did. We kept trying different inputs, getting increasingly creative with order, special characters, Unicode characters, other languages as we progressed. But we never chose a different path. Even as I tried to encourage us to take notes so we wouldn't try the same things we'd already tried, we didn't.

We did manage to find a good resource for copying and pasting unicode characters, but we didn't learn how to explore the application more efficiently, or take what we learned earlier in the session to apply it to the rest of the session.

The Power of Exploratory Testing

Brute force will get you somewhere. Trying enough different inputs, or different pages, and you'll gather more information about how the application works. But the power of exploratory testing comes from learning from your earlier results. It's realizing there are different ways to go, different paths to follow, jumping on one of those while it serves you, and making sure everyone else is along for the ride.

When you say no to what you don't have the energy for, you leave your time, attention, and devotion free to pursue what keeps you energized. Through a tumult of plans and dreams that only 2020 could continue to crush, the Agile Testing Days crew put on a revitalizing conference. I only caught a glimpse of the preparation that went into providing a top-notch experience for speakers and participants, and I have to commend the group on their commitment to the cause. Bravo.

The talks were recorded. I've decided to cut myself off on catching up on missed talks after two weeks. Reflecting on what wisdom I've garned from a small subset of the offerings at Agile Testing Days will be more valuable than being exposed to every last word. In that spirit, let me give you a few takeaways in place of an exhaustive recap or analysis.

Parveen Khan: watch this if you're looking to collaborate

• It is in doing the work that we discover what we must do.
• Don't get stuck when learning something alone; pair.

João Proença: watch this if you need a different direction

• Performing a health check and making a diagnosis are different skills.
• What barriers do we set for ourselves?
• The product is all the pieces the customers see, including the cloud infrastructure (and whatever else your team isn't responsible for).

Paul Holland & Huib Schoots: watch this if your automation isn't providing valuable information

• The sunk cost fallacy means we're unlikely to get rid of automation code, even if it's ineffective.
• We often fail at getting people to change.

Gitte Kligaard talk: watch this if you've been hiding your true self at work

• "I'm just going to rewind." (Possibly my favorite takeaway from this conference, I'm going to use this anytime I trip over my words.)
• Creativity comes when inspired by others.
• Being professional is knowing your craft, and admitting when you don't know.
• Spend time with yourself to listen to yourself.

Alex Schladebeck & Ashley Hunsberger: no recording, but follow these ladies on Twitter for how to balance life and leadership

• Tell people what you want. They may be able to help you.
• Have a clear vision of what you want.
• Write down your fear. Ask yourself: Why is this positive? How can I build the courage to do this?
• Be explicit about what you're doing.
• Show how you're working.
• Build a practice of reflection.

Angie Jones: watch this if you need a push to get into automation

• Determine your goal first. If you don't know it, you'll definitely fail.
• When you're a leader, celebrate the small victories.
• There's no reason to shame people for being creative and doing the best they can.
• It's not realistic to assume master level by default.
• Find out why developers don't participate in automation.

Rob Meaney: watch this if you don't get why observability is important

• We learn profound lessons from painful experiences.
• Build relationships. Influence people at the right time.
• Pain + reflection => progress
• It doesn't matter how much we test a thing if nobody wants to use it.

Tobias Geyer: watch this if you're struggling with an ethical dilemna

• Could ethics be a non-functional requirement?
• Read the codes of ethics proposed by the IEEE, ISTQB, ACM, and James Bach.
• A hippocrattic oath for software testing: avoid harm.
• Promote environment sustainability.

Smita Mishra: watch this if you're interviewing users

• Listen to be able to ask clarifying questions and dig deeper.
• Ask users: their objective, what they found, about the impact, where they're struggling, what could make their lives easier.

Eveline Moolenaars: watch this if you're learning to coach

• "We have a policy; it's somewhere on the internet" isn't enough.
• "Everyone deserves a coach to make them aware of what they've forgotten." ~Brad Gilbert

Federico Toledo: watch this if you or those around you are losing steam

• A tester with a sense of purpose is more resilient.
• Focus on strengths more than weaknesses, turning up the good.
• Provide visibility that you're doing something with the feedback.
• Ask people if they're getting what they need.

Nicola Sedgwick: watch this if you're the gatekeeper for quality

• Do not correlate your own successes with the quality of the entire system.
• Contract-driven work is not holistic quality.
• Report when there has been no progress.
• Ask for a code walk-through before a build is ready.
• Shift quality left all the way to the executive team.
• Testers were hoarding responsibility for quality. Let it go.

Joep Schuurkes: watch this if (or while) you're stuck in an ineffective meeting

• Ask yourself: is my work inside or outside of meetings?
• Get the right group of people in the room before holding a meeting.
• Express specific appreciations for things done well so people will keep doing them.
• Meetings are synchornous collaboration with a purpose.
• Standups should remove impediments.
• The metaphors we choose say something about how we feel about our work.
• Leadership is creating an environment where everyone can contribute to solving a problem. ~Jerry Weinberg

Clare Norman: watch this if you're truly stuck

• Quality is everyone's care.
• "Success is liking who you are, what you do, and how you do it." ~Maya Angelou
• Your courage quota for the day might vary.
• You don't get better by doing the same thing everyday.
• I didn't know how I needed to be helped.
• You can't drag someone through time.
• We live far less in the present than we ought to.
• How magical is it when people value the change that you're making?
• Passion spreads when other people share in your excitement.
• Have a cheerleader in your life.

Clare Norman channeling Maya Angelou on success

Jenny Bramble: watch this if your tests are failing on expected behavior

• Many defects I filed are never fixed. Unfixed defects become expected behavior.
• Automation has never found a defect. Automation tells us behavior has changed.
• Tests should pass on expected behavior, including some defects.
• Document your tests for others, including your future self.

James Lyndsay & Anne Colder: watch this if you feel like an impostor

• Acknowledge when you can't help.
• If you could take one step forward, what would you do?
• Talk to your person, or ask a smaller group of people if they have an answer to your problem.
• Forgiveness is one of the most powerful things you can do as a human being.
• After you make a mistake: wallow in it (really feel it first), forgive yourself, then dance.
• Your job title allows other people to figure out how to work with you.

Gitte Klitgaard AMA: watch this if you miss the hallway track

• Listen to hear what's being said, not to respond.
• We can't read minds.
• Creating a safe space allows you to feel uncomfortable.
• Silence is a tool.

Sophie Küster: watch this if you're ready to tell your colleagues your big secret

• There is strength in showing vulnerability.
• People want to help you. Let them.
• Ask yourself: how am I treating myself?
• Anything worth doing is worth doing poorly. Poorly done is better than not done at all.

Gitte Klitgaard & Morgan Ahlström: watch this if you want to get psychological safety in the boardroom and on the roadmap

• Make time to address psychological safety. Put it on the same level in your team's goals as your product goals.
• We are role models. We lead by example.
• More people were uncomfortable giving than receiving code reviews.

Ard Kramer: watch this if you're getting burned out

• Humans have an unrealitic belief in our own influence.
• Performing your testing role doesn't make you popular.
• Most people looking for confirmation that the software is working. To form a logical proof, we look for evidence that the software doesn't work.
• Ask yourself: which circumstances could I control? Did I manage expectations correctly?

I'm grateful I got to connect with people who keep me going. I'm grateful Agile Testing Days tried to make this work in person. I'm grateful I was able to drop my workshop instead of abaondoning people in breakout rooms to do something they've never done before. I'm grateful the mobbing session I helped organize and facilitate went smoothly. I'm grateful that my job allows me the time and space to be rejunivated by this all. I'm grateful. 🙏

In his crafting time, one of our developers decided to fine-tune our search results. He added relevancy scoring to give weights to different text fields. It was my job to determine if the right results were turning up at the top of the list of search results. So I had to ponder: what made a search result relevant?

First, I realized that feedback from our users is the best way to answer this question. Anything I could do to get this feature out into production, where we'd get real data about what people searched for, would be more valuable that brainstorming test cases for this feature. I set myself a timebox of two and a half hours, the rest of the afternoon on a day in a week filled with competing priorities. We'd agreed as a team ahead of time that I could determine the testing approach, and our product owner would decide what was or wasn't worth fixing before this feature went out.

I saved the first 45 minutes of my timebox to research what people had to say about search relevancy. Surely I was not the first person contemplating this problem. Over on the Ministry of Testing Club forum, I found what seemed to be a promising title of a post. It turned out a past Elizabeth from a year ago wrote it, and nobody had answered it satisfactorily in the intervening time. 🤦‍♀️

After Duck-Duck-Go-ing some actual answers, I found a couple of resources from websites I've trusted and found fruitful in the past: A List Apart and philosophe.com. A List Apart suggested honing in on a specific intent, searching for the queries users seek most frequently, and seeing how far those items fall from the top in the search results. The philosophe guidance about testing search gave me something deeper to consider: users shouldn't have to ponder the reasoning behind the search results. That was enough for me to develop some test cases.

As I searched and adjusted the weights of various fields, plenty of normal things happened: setting the relevancy values to zero meant the result wasn't listed, multiple instances of the same word counted more than a single instance, and giving fields stronger weights caused their results to move up in the rankings. But as a bug magnet, I uncovered things that were both interesting to discover and outside the original scope of the story.

Log of zero is negative infinity

Bugs

1. I opted to edit data that was already in our test environment rather than setting up completely new data. In doing so, I discovered a couple description fields that were missing an indexing update in the search results when edited through the UI.

2. I tried to use the default values for everything to see "normal" results. One field was going to add a value to the relevancy rating, so zero seemed like it should be the default option. Unfortunately a couple of the options for the weighting feature transformed the value using the log and ln (natural log) functions, which are undefined at zero. All my search results disappeared.

3. I looked at the data that was already in the database, and used a search term that showed up a lot already. It turned up nothing. I searched for part of the word. That turned up all the results I was expecting. I realized the search term was indexed separately because of the characters we used to break the word apart. Imagine having a bunch of sun-bleached items, but you can only find them if you search for sun or bleached, not sun-bleached.

Bug 1 the developer agreed was a bug, and we fixed as part of the story. Bug 2 was "working as expected" from a developer point-of-view, but seemed a little weird to the product owner. We meant to look into as part of the story and decide if we should eliminate the log functions as options entirely, but other priorities came crashing down upon before we could. It's out on production to the handful of internal users with access to the relevancy tuning. Bug 3 we added to the backlog, and I hope someday a swarm of user complaints make it a priority for us.

Morals

1. Users know best.
2. Someone else on the internet has had your problem before.
3. Report information that matters.
4. When the risk is low, let it go.