Tesco took its own IT developments into its own hands eight years ago after it became apparent that more than a thousand different third-party systems and services were significantly limiting development opportunities for online store and shopping operations, and introducing innovative features. The company has reorganized the internal developments in the Tesco Technology sector, which today, with more than 4,500 employees worldwide, upgrades the IT infrastructure of the store chain to modern technologies. In addition, about a hundred highly qualified software engineers work from the development team in Budapest, whose developments were first honed in Great Britain and then “come home” after testing, that is, to Tesco stores in Hungary and other countries of our region, such as Slovakia, the Republic of Czechs.
The in-house infrastructure offers many advantages, making shopping not only more convenient – thanks to self-service payments, for example – but also more secure. The prevention and control of losses, alerting to developments in Budapest serves this purpose, thanks to which the status of all cash registers can be monitored in real time. In this way, experts can immediately intervene in the event of any problems or malfunctions, even if a customer inadvertently or unintentionally reads a product using the corresponding barcode during self-service checkouts.
Up to the last payment, direct
For all this, Tesco Technology has developed a very powerful monitoring and alarm system – a well-scalable architecture is also needed, since more than 70,000 devices must be continuously monitored in real time. The task is not facilitated by the need to monitor the status of at least 30 container services per device, which are sampled and reported every 15 seconds.
To collect metrics in the case of the project’s MVP, the development team uses the open source Go language written in Go, which works with time-series-based metrics – browsing banners would not be ideal for real-time data mining. At the same time, it can already be noted that in the long run, Prometheus, famous in many areas, will not be able to use it effectively either, due to memory starvation. At the current development stage, the service will continue to scrap about 13,000 data recovery devices, using a software stack with 1 TB of memory. To recharge this, the system setup time can be up to half an hour.
So the company is already transferring data from Prometheus to VictoriaMetrics, and later moving away from the previous device – VicotriaMetrics promises a more efficient horizontal scale with up to seven times less memory usage. Furthermore, the switch is greatly simplified by the fact that VictoriaMetrics maintains at least the same amount of friendship with Grafana, which is used for data visualization as Prometheus, so it can be positioned to work effortlessly as a “surrogate” without having to do a lot of extra stuff configuration tasks.
NET in the network
Of course, for metrics to be transmitted efficiently, they must also be embedded in an appropriate network, but this is not certain everywhere – in the mountains of Scotland, for example, there is particularly poor bandwidth available to stores. Therefore, it is very important to correctly prioritize the various data streams, and even ensure offline operation, if necessary.
Here, software engineers and network experts at Tesco Technology have found the solution in open source C++ written with a C++ proxy. With the latter, instead of layer 7 (application layer), priority is solved at layer 4 (transport layer), setting rate limits so that in the event of a software update, the network of the particular store will not be overloaded by delivery of containers up to 3-400 Mega for cash registers and for different devices. This ensures that payments and authentication are always given the highest priority. Achieving this requires very intense collaboration from teams of development, operational, and network experts.
And of course the data communication is not only between the hardware, the cloud and the central servers of the stores, that is, the servers of the store platform. In the store device mesh network, the different devices also communicate directly with each other. One of the most common examples of this is the purchase of alcohol at a self-service checkout, in which a cashier notifies a colleague who oversees a particular checkout line that an age-restricted product has been placed on the surface of the package. He can then decide whether it is necessary to request ID from the customer, or whether his Frost Pagoda is sufficient to prove the appropriate age. But machine vision devices based on Nvidia Tegra GPUs that monitor what merchandise is being read at checkout also communicate with cash registers in a similar way. Hardware in development will verify that the customer is actually trying to leave with two pounds of bananas, or just a game console of the same weight.
It is better to be afraid than to be afraid
Elimination of such losses is also an essential function of chain-owned systems, for which the loss prevention team is responsible. To develop a cash vision monitoring machine vision solution, the company has assembled a dedicated computer vision team. Developers will first test self-taught machine vision algorithms in a lab environment and later in the field, which will work in tandem with several risk analysis algorithms to determine when it’s worth intervening in the event of a potential suspicious payment.
At the same time, self-service cash register scales are getting smarter, and learning weighing databases is behind them. If the product is not read correctly, the balance does not measure the weight recorded in the database, it will notify the close colleague who will check the product. If the employee finds that all is well, he confirms that the correct product has already been read, but has been changed in weight by the manufacturer. The database will then continue to work with the new information. Of course, there are a lot of challenges in writing the corresponding algorithms, as the differences may be due to the error of the mate or the weight cell sensing cell – these situations must also be recognized.
Not only the developments themselves, but also their delivery is an easy task, which is why the company has built its own roll-out, with a complex pipeline so that the work of a 30-40 team can be synchronized and made continuous and new services that can be gradually distributed through different stores. For tens of thousands of lockers. After new software versions are registered, images associated with them are first uploaded to the central servers of the above-mentioned stores, usually at dawn, which then gradually distributes updates to store checkouts. All this is done with proper prioritization of the data concept so that the purchase process can also remain smooth in the selected store.
The above Tesco Technology developments, which are behind the Budapest team, will be honed in the UK and Ireland this year, and at the end of 2022 and the beginning of 2023 we can already meet them in Tesco stores in Hungary.
The local Tesco Technology team is constantly expanding, the company is mainly looking for “V-shaped” developers: for example, in the case of a Java developer, the “V” tip indicates a deep knowledge of Java, while the broad stems contain many horizontal competencies. These can be DevOps or even business presentation and communication skills, with which they can effectively communicate the globally applicable developments of Tesco Technology in Hungary – even through remote work, which the company supports to the maximum.
[A Tesco Technology megbízásából készített, fizetett anyag.]
“Social media evangelist. Baconaholic. Devoted reader. Twitter scholar. Avid coffee trailblazer.”