Bloggers Club October 2022 - Trick or treat - Irreproducible Bug

 Let me tell you a spooky story about an irreproducible bug. It took a place a long time ago, before seven companies, and seven projects, there stood an irreproducible bug! At the time this happened I was working on a fairly large enterprise project,. The whole system was made up of various applications, written in a few different languages, over time, by many different teams. 

The core defect was a bit of an edge case, where just users with certain country codes would not get eCommerce promotional emails sent to them. It was a wild goose chase, I was the first to investigate the bugs, as I was in the application support team at the time, about 20 hours per week. I tried in different environments, but to no avail - I could not figure it out. Used a test harness to trick the dates, as it was a kind of email that gets sent 24 hours after registration. I got very stubborn about it and spent about 4 hours without getting up from my chair, trying to figure out the root cause. Why was this email not sent in production? 

It took over a week and a whole bunch of people to figure out what the issue's root cause was some obscure configuration setting if memory serves me right. The one thing I learned from this scary experience was that being persistent is sometimes not the most efficient thing you can do. Instead of going along into the haunted legacy system, it is better to call for help! I was pretty green at the time and if it took a team of experienced seniors a week to figure this one out, it would have taken me months - and that is not something that business can say yes to!

Thanks for reading, I hope it wasn't too scary!