Python Profiler Links to AI to Improve Code

Python continues its dominance as one of the top programming languages, yet it’s notorious for being sluggish, so some software developers resort to profilers—tools that run programs to determine which parts of the code are slow. But profilers can be frustrating, because they can slow programs down during the profiling process, as they use a lot of resources to measure memory consumption, runtimes, duration and frequency of function calls, and other metrics.

Researchers at the University of Massachusetts Amherst aim to change that with Scalene, a Python profiler that not only identifies inefficiencies in code but also recommends how that code can be improved, with the help of AI.

Scalene simultaneously profiles usage of three components of hardware: the CPU, GPU, and memory. To make Scalene faster than standard profilers, the team—which includes graduate students Sam Stern and Juan Altmayer Pizzorno—opted for sampling-based profiling for both CPU and GPU. “It wakes up periodically, looks at what is executing at the time, records that information, and goes back to sleep again, so it’s a lot less disruptive,” says Emery Berger, a professor of computer science at UMass Amherst. The GPU profiler reports GPU time. The CPU profiler does something a bit more sophisticated. Python code uses an interpreter to translate the code into machine code line by line as the program runs. Other “native” code, is compiled into machine code all at once. The profiler separates time spent on Python code from time spent in native code.

“What Scalene does is it finds the hot spot or region of code that seems to be taking the most time or other resources and sends that to OpenAI with a prompt”—Sam Stern, University of Massachusetts Amherst

Scalene’s memory profiler implements what Berger calls “high-water mark” sampling. He likens memory usage to an ocean wave that rises and falls, with Scalene recording and keeping track of the height of the wave. As long as the wave doesn’t crest too high or tumble too low, Scalene doesn’t do any additional accounting. Only when the wave crosses a high-water mark threshold does it record and track continuously.

Additionally, Scalene’s memory profiler employs a sampling algorithm that distinguishes between memory consumption stemming from an underlying native code library versus the Python interpreter itself. This process unveils instances where coders can switch to native code to improve performance. The memory profiler automatically pinpoints lines of Python or native code that are likely to result in so-called memory leaks. A memory leak happens when a program allocates memory, uses it, then neglects to free it. This can affect a program’s performance by reducing the amount of memory available. Another metric called copy volume is tracked as well. It spots costly and often inadvertent copying operations that occur when Python converts between native and Python data representations.

Scalene’s techniques lead to an average overhead of 2 percent for CPU and GPU profiling and 30 percent when including memory, which puts the profiler in line with other profilers having low to modest overhead.

A Little Help From AI

What makes Scalene stand out from its peers is its integration with AI. Through its Web-based UI, the Python profiler shows a line-level and per-function breakdown of CPU and memory usage, and with the click of a button, it sends the code to OpenAI’s GPT-4 large language model, asking it to produce an optimized version of the code.

“What Scalene does is it finds the hot spot or region of code that seems to be taking the most time or other resources and sends that to OpenAI with a prompt,” Stern says. So aside from identifying where and why the code is inefficient, the Python profiler taps into the power of AI to suggest ways to make the code more efficient.

Scalene’s GPU profiling works only on Nvidia-based systems, and for Microsoft Windows, it only does CPU and GPU profiling. Despite these limitations, the Python profiler is now in broad use, with 780,000 downloads to date and users sharing their success stories of Scalene helping them improve the performance of their code.

The next step is to exploit the changes in Python’s newest release to further improve Scalene’s profiling, boost its speed, and make it even less intrusive. The team is also looking to enhance how the profiler works with AI. “What we’re doing is improving on the AI-generated optimization that we have right now to check that the suggested code works, is right, and is better,” says Stern.

But what Berger ultimately envisions is for Scalene to be an optimization and proposal assistant—one that guides software developers to enhance the performance of their programs. “A profiler that tells you the parts where the code is slow is great, but it may be really hard for a programmer to do something with that information,” he says. “Being able to close this loop between what the tool can provide and what the programmer actually wants is exciting and something we’re trying to take advantage of in any way we can.”

Berger and his team presented Scalene in July at the 17th USENIX Symposium on Operating Systems Design and Implementation, in Boston, where it was awarded best paper.