This article is for all the developers and coders out there who are working hard on various projects. Investing time in building a good error-handling logic is of utmost importance and should not be ignored.
The following are the events that unfolded with our team and taught us a lesson that we would never forget.
An Email from WordPress Security Team
It all started with an email from the WordPress security team regarding a potential security vulnerability in an older version of QSM.
We immediately took action and followed our standard procedure to patch the vulnerabilities.
We made the required changes to our code and sent the changes to our QA team as per the procedure.
We informed our QA team regarding the vulnerabilities and the code changes that might impact some database queries so that they can test the relevant functions and features accordingly.
The QA team tested the plugins thoroughly and everything worked just fine after which they gave us the green light to go ahead and release QSM 7.1.13
However, a couple of days later we received a ticket from one of our customers complaining that their quiz has stopped working after updating QSM to the latest version.
This time our support team swung into action and tried replicating the issue on our test servers. But everything was working fine, our old quizzes were working just fine as well as any new quiz that we created using QSM 7.1.13
We started investigating more about the issue by talking to an affected customer and asking for further details. Meanwhile, our engineering team tried simulating the update from old QSM versions. We went as far as simulating an update from QSM 6 to the latest QSM.
More Tickets by the Affected Customers
Despite all our efforts, we were not able to get any breakthrough. Meanwhile, we got another ticket from a customer complaining that their questions got deleted from their quiz.
We started the same process to investigate the issue described by this customer. We were not getting any luck as more tickets started coming complaining about similar issues.
The issue became our top priority and our engineering team and support team were working hard to find a clue. We revisited the changes we had made in the last version to find any link with the ongoing issue, but all those changes appeared to be harmless.
We tried replicating on different web servers and MySQL servers but we failed every time to replicate the issue. Soon more engineers got involved and tried to narrow down the possibilities but everyone ended up with similar conclusions.
Meanwhile one of our support engineers found a magical workaround to bring the quizzes back. One has to simply press any save button on the quiz admin screen and all the questions in that quiz would restore.
As crazy as it may sound but it worked. At least we had found a way to help our customers for the time being.
The Eureka Moment!
There were still many questions that needed to be answered. We had almost hit a dead end with our investigation when suddenly out of the blue, we received a pull request from a contributor on our git repository.
The developer wanted to remove { and } from SQL as it was generating the following error in their MySQL setup.
[26-Mar-2021 01:34:55 UTC] WordPress database error You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '{ } { } {ORDER BY question_order ASC} { }' at line 1 for query SELECT * FROM wp_qyxhycyyyb_mlw_questions WHERE quiz_id=10 AND deleted=0 { } { } {ORDER BY question_order ASC} { } made by require('wp-blog-header.php'), require_once('wp-includes/template-loader.php'), ...
This was exactly the breakthrough we were looking for so desperately throughout the week. We continued our investigation in this direction and asked the following questions from this contributor.
What Questions Did We Ask?
- What is the initial version of QSM you have installed?
- Are you facing this issue with the older quizzes?
- Are newly created quizzes working fine?
- Could you see the questions on the edit quiz page but they are not appearing in the frontend?
We soon received a reply with the answers to the said questions and made the required fixes. At last, we were happy to finally mark the issue as resolved.
What Lessons Did We Learn?
We found out that the issue was affecting quizzes that were created a long time ago. The reason why we couldn’t replicate the issue was because of a conditional statement that executed the buggy code only under very specific circumstances.
Moreover, the code was failing silently without displaying any meaningful error messages to help the users or even our support team members.
The biggest takeaway for our team is to improve our error handling logic so that we can quickly identify the potential cause in such situations going forward.
We will be improving our error handling and logging in upcoming releases.