Sergey Bevzenko, a senior software engineer at Yandex, explains how software quality control affects the functioning of businesses, the state and overall society.
GVS: The Cost of Poor Software Quality in the US: A 2020 Report released by CISQ (Consortium for Information & Software Quality) indicates that in one year alone, the cost of poor software quality related to failures in the US totaled $1.275 trillion. Impressive figures, aren’t they? When we take into account not only money but think more widely — about different aspects of society — what is the impact of software quality on those?
Sergey Bevzenko: The impact is tremendous. Development and operation of information technologies’ errors can lead to catastrophic consequences such as serious equipment malfunction and can even be fatal for some individuals. For example, in early 2023 alone, the market capitalization of Alphabet, company owning Google, fell by $100 billion after a bug was occured in the operation of its new artificial intelligence technology (chatbot).
And there are plenty of similar examples from different industries and countries. It happens that not only citizens or businesses are directly affected, but also the state. In the mid-2010s, I think in 2015, there was a software glitch in the United Kingdom, which caused the Bloomberg terminal’s breakdown. The country’s government was unable to sell the debt — we were talking about billions of pounds.
GVS: How can we prevent such negative events?
Sergey Bevzenko: All those participating in the process are responsible for the quality of software: both managerial staff and technical staff (testers, developers). In fact, every field has quality protocols. There are lots of levels of testing and inspections at various stages of software development and release. However, mistakes still do happen.
Curiously, there are some areas where serious violations happen much less frequently. For example, Voyager-1 and Voyager-2 were launched more than 40 years ago, but their software still keeps working. And yes, breakdowns happen there as well, but what is interesting is that each one of them can either be fixed automatically, or the craft can continue to operate despite them. By contrast, in other areas there are far more failures.
GVS: Does this mean that the cost of a bug in software is different depending on the industry?
Sergey Bevzenko: It looks like so, yes, that the price of an error is lower in some cases. But who is setting this price? In my opinion, the answer is quite ambiguous. For instance, a mistake occuring in a computer game would lead to the point when a gamer in a competition wouldn’t be able to complete the level and will lose money. This, at first glance, does not seem to pose as much of a threat to the individual and society as software malfunctions in medical equipment. However, does this mean that one can be less diligent in quality control? Assume that a programmer initially developed computer games and was negligent in testing. If he moves to another company and starts creating software for airplanes or medicine, will he be able to break the habit of lax quality control?
When I worked at Pegas Touristik, a tour operator, my team and I paid special attention to testing, especially to its automation. Although the company’s main business was tourism rather than software development and sales, maintaining high quality of all information systems was essential. Otherwise, people would not be able to buy airplane tickets in time, learn about canceled flights, which would make them lose time, money and in return the tourist operator would be mired in lawsuits. Nothing similar had happened to the company on a larger than average scale, because we as the IT department tried to do our job properly.
GVS: You were involved in software development in large companies: “Avito” (ranked #1 in SimilarWeb’s ranking of most visited websites in November 2021), “Yandex” (Fortune’s top 100 fastest growing companies in the world in 2019), Ozon Group (included in Business Insider’s 4th annual ranking of the world’s 100 largest IT companies; leader in Russian-language online retailers for the period from July 1, 2021 to June 30, 2022). In such large organizations, do approaches to software testing differ from those in small businesses or startups?
Sergey Bevzenko: Both yes and no. Quality control standards are identical for everyone, so both a startup and an industrial giant are able to perform high-level testing. But obviously what tends to differ is their approach to the organization of testing processes. As a rule, in small companies and startups one single team is responsible for a large block of functions.
When it comes to large companies, a lot depends on whether the company specializes in information technologies or whether its business belongs to another sphere. For instance, I worked for Pegas Touristik, a large CIS travel operator. It is a large organization that has its own airline called Nordwind Airlines, hotels, and offices in different countries such as ones from the former Soviet Union, Turkey, Egypt, Thailand, Vietnam, China, the United Arab Emirates and Israel. My responsibilities included managing the website maintenance and its integration with the Global Distribution System (GDS). Pegas Touristik is such a big international company, however its main business is not the development and sale of IT products. Therefore, the IT department at Pegas Touristik worked solely to support its needs, and the organization of software testing processes was similar to one in a small business.
GVS: Can you tell us about any of the recent innovations you have come up with to enhance software testing capabilities?
Sergey Bevzenko: I am a maintainer of an Open Source tool for load testing called Yandex Pandora. Load testing is used to evaluate the performance and reliability of a product by means of load testing software which is subjected to significant loads while running it.
My responsibilities include maintaining the Yandex Pandora tool, improving performance, and fixing bugs. I also develop new functionality and work with external contributors.
One more important task of mine is to create complex scenarios in load testing using simple configurations. I have successfully elaborated this logic and presented it at several conferences.
GVS: Your competencies include the ability to develop technologies for load testing, which is one of the most complex and expensive types of software quality control. There are not many such engineers. How did you become that skilled in such a difficult segment of information technologies?
Sergey Bevzenko: I graduated with honors from the Far Eastern State Technical University (modern name — Far Eastern Federal University). After completing my bachelor’s degree, I then graduated with a master’s degree in engineering and technology in the field of automation and control. After that, I worked for two years as an engineer in a field unrelated to a programming one. However, information technology has always attracted my interest, so I left my position at a radio plant and joined the IT department of a large company.
I enjoyed working in a large company straight away, because the functionality was challenging, engaging and contributed to my professional growth. I always tried to undertake the most difficult tasks, because only by those means you can become a really good specialist. I grew as a professional and was subsequently hired only by top companies, market leaders: Delivery Club (formerly owned by the German company Foodpanda, then sold to Sberbank), Ozon Group, Avito, and now Yandex.
When I joined Yandex, the company set a goal — to create an Open Source load testing tool. I was excited to develop it because such technologies can make load testing more accessible, and thus enable more companies to build high-quality software.
GVS: Regulators in various countries are concerned about the costs incurred by governments, businesses and society due to product malfunctions in information technology. The European Commission implemented the ICEBERG project to develop a strategy to improve quality in software development. The National Institute of Standards and Technology (NIST) together with the U.S. National Security Agency has developed recommendations on what minimum standards for software testing should be used. China and many other countries have similar standards. What is the role of regulatory bodies in improving software quality?
Sergey Bevzenko: I would rather discuss here the importance of team work among regulators and professional associations in the IT sector and not that much the role of regulatory bodies if we analyze them separately. It is extremely important for society to receive quality software products.
If regulators don’t listen to the professional community, rules and laws can become non-viable and slow down the progress of technology. When on the contrary governments, ministries, state agencies go on about the professional community’s opinions there is a high risk of monopolization of the industry by large companies and in essence a huge risk that the laws will only be used for the interests of business to the detriment of the society. Therefore, I would say that dialog is essential here, and all parties should be involved and have a word in it. But surely we do not take only regulators and IT professionals into account but also society in the broader sense like ethics committees, lawyers, human rights activists and everyone who is not indifferent to making technology work for the benefit of people.