The uArch 2026 Cohort

The 8th annual Undergraduate Architecture Mentoring (uArch) Workshop took place alongside the International Symposium of Computer Architecture (ISCA) 2026, in Raleigh North Carolina. Each year, uArch introduces undergraduate and early master’s students to research and career paths in computer architecture through technical talks, sessions on graduate school life, and mentoring panels on graduate admissions and navigating the research landscape.  

More importantly, uArch is about connecting students with the broader computer architecture community. Students meet researchers from academia and industry, learn about their experiences, and get a glimpse of the community they might one day join. Besides the uArch workshop itself, students also participated in the Meet the Senior Architect program and attended the main ISCA conference, giving them the opportunity to engage with this community throughout the week.

For many students, these interactions were one of the most memorable parts of their experience. As Mihir Mahesh, an attendee from the University of Maryland, put it, “It was very interesting to see how different professors and industry researchers at varying universities and companies formed a community with one another under the shared interest of research.” Similarly, Jerry Huang, from UCLA, described the welcoming environment as a big factor in making ISCA a positive experience for students like him, where it was easy to meet and strike up a conversation with everyone from undergraduates and graduate students to academic researchers and industry architects. 

For Abby Quesnell, a junior at the University of Wisconsin-Madison, uArch also made her “more aware of [her] career options and what some of those different paths look like for industry vs. PhD vs. Master’s.” For some students, uArch helped clarify their plans for graduate school. Vadym Musiienko of Pomona College, who is applying to PhD programs soon, said that uArch “showed me exactly what I’m signing up for, which made it easy to commit.”

In this post, we take a closer look at how uArch is organized and what the workshop looks like from the student perspective.

Outreach

Organizing uArch starts well before the workshop itself. Each year, the organizing team reaches out to faculty in computer architecture and systems to share the workshop with their students and encourage applications. We also work to expand outreach to universities in regions connected to the ISCA host location. We are especially grateful to the ISCA 2026 General Chairs for their continued support in helping us reach students and organize this year’s workshop. 

For uArch 2026, more than 80% of in-person attendees came from North America, representing 30 unique schools. We also offered two types of travel grants to help make participation more accessible. Matching grants allowed faculty or universities to contribute toward a student’s travel costs alongside uArch funding, while full grants covered the costs through uArch funds. 

Attendees

As in previous years, we were thrilled to receive many impressive applications from students around the world. With limited funding and capacity, we ultimately welcomed 41 students in person. As part of the application process, students submitted a statement of interest describing their background, their interest in graduate education, and what they hoped to gain from uArch. The organizing team reviewed these applications to build a cohort of students with diverse backgrounds and interests. 

Pie chart showing the countries represented among selected in-person applicants to uArch 2026. 

Pie chart showing the schools represented among selected in-person applicants to uArch 2026. 

While our cohort spanned a wide range of institutions, significant opportunities remain to broaden global participation, particularly among universities outside North America. Expanding that reach will require continued outreach and support from the broader computer architecture community. 

How can the community help

Sponsor more matching-grant students: Matching grants allow faculty and universities to share the cost of bringing students to uArch. If you know talented undergraduate or master’s students who are excited about research, consider supporting their participation. 

Industry sponsorship: We are grateful to our sponsors, ACM SIGARCH, Google, AMD, Cerebras, and Microsoft, as well as the faculty members who contributed to the matching-grant program. Additional support from industry and academia would allow us to bring more students to uArch and broaden participation. 

Join the organizers club: uArch is always looking for people interested in helping organize and grow the workshop. Organizers contribute to application review, programming, logistics, and session facilitation. We also welcome graduate students interested in serving as mentors before and during the workshop. 

uArch 2026

This year’s uArch workshop opened with a keynote from Professor Chris Fletcher of UC Berkeley on new directions in computer architecture in the age of agentic AI. For Vadym, hearing researchers reflect on how AI is changing their work was particularly memorable: “Hearing working researchers say how they’re adapting to it, and what still worries them, wasn’t something I expected going in.” 

For several students, the keynote also changed how they thought about getting started in research. Abby described how AI could lower some of the barriers that make research feel intimidating to undergraduates: 

“Research as a general topic is intimidating because most research papers are highly technical. It seems difficult to break into that level of knowledge and create your own new ideas. AI makes that barrier seem lower, as it can help guide you through research.” 

Mihir summarized another key takeaway from the discussion: “The limits of research are not in technical skills and experience, but rather in ideas.” 

The program then moved into three panels: “The Route to Graduate School,” “Life in Graduate School,” and “Life After Graduate School.” The panels gave students an opportunity to hear directly from faculty and graduate students about applications, graduate life, and careers.

For students like Jerry who are considering graduate school, the application process can feel overwhelming. As he put it, “Hearing from the grad students about their experiences so far, and from the professors about applications, helped break down the process … and ease some nerves.” 

For Mihir, the “Life in Graduate School” panel “put grad school in a much more realistic light, as the panelists shared a lot about their own lives, both the good and bad.” The discussion showed him that “a PhD is not as linear or straightforward as it seemed to [him] initially,” and that everyone has their own timeline.

The workshop concluded with 15-minute round-robin office hours, where students met with faculty to ask about research, careers, and graduate recruitment. We are grateful to the faculty members whose time and perspectives made these conversations possible. 

For Jerry, conversations with Professor Joel Emer and Professor Callie Hao clarified the connection between computer architecture and chip design. Interested in both areas, he was glad to hear that “I won’t have to choose – neither are escapable if I want to be a proficient architect or chip designer.” 

For Vadym, the conversations also offered a chance to learn about the people behind the research: “I got to hear how they got where they are, not just what they work on. All of them were honest and generous with us, and a few of those conversations could turn into a future advisor.”

For the full schedule and details, visit the workshop program page. 

Support from the Community

We greatly appreciate the community’s continued support. uArch would not be possible without the time, effort, and enthusiasm of the keynote speakers, panelists, faculty members who held office hours, student mentors, and ISCA General Chairs. This collective effort allowed us to reach students from around the world. We extend our sincere gratitude to everyone in the community who helped make this year’s uArch workshop a success. 

About the Author:

Irene Wang is a PhD student at the Georgia Institute of Technology. Her research interests focus on designing computing systems to enable scalable and efficient AI datacenter infrastructure. 

We would also like to thank the students, Mihir Mahesh, Jerry Huang, Abby Quesnell, and Vadym Musiienko, who volunteered to contribute to this year’s blog post. Their reflections and perspectives give us a chance to see uArch through the eyes of the students it is designed to support.

Disclaimer: These posts are written by individual contributors to share their thoughts on the Computer Architecture Today blog for the benefit of the community. Any views or opinions represented in this blog are personal, belong solely to the blog author and do not represent those of ACM SIGARCH or its parent organization, ACM.

Lessons from using an AI coding agent in LLM systems research

This project began with a routing problem. When an LLM request arrives, the system must decide which GPU will execute prefill and which GPU will execute decode. The same GPU can execute both phases. Alternatively, a prefill-only GPU can execute prefill and transfer the request’s key-value, or KV, state to a GPU that executes decode. Local prefill avoids that transfer, but it can lengthen the decode iterations of requests that are already generating tokens on the same GPU. Remote prefill keeps that work off the decoder, but it can add queueing at the prefill-only GPU and delay the request while its KV state is transferred. We wanted a router that considered both the arriving request and the requests already in service.

An AI coding agent made it possible to test this idea quickly. It added instrumentation, generated experiments, replayed the same arrival after forcing different decoder and prefill-path choices, and summarized thousands of measurements. This speed made broad exploration practical. It also allowed a weak experiment to produce a large body of convincing results.

That happened in our project. The agent helped implement the routing rule and produced tidy, reproducible results, but some of those results were scientifically wrong. We later used the same agent to trace metrics through the code and replay the decisions that did not make sense.

The project taught us to give the human, the agent, and the simulator different jobs. We decided what a metric meant, what information the router was allowed to use, and what made a comparison fair. The agent turned those requirements into code and checks. The simulator recorded the queues, batches, and completion times. We used that record to decide whether a result supported our claim.

Local and remote prefill paths

LLM serving has two computational phases. During prefill, a GPU performs the computations over all input tokens and builds the KV state needed to generate output tokens. During decode, a GPU generates the response one token at a time. A GPU that executes both phases is a mixed GPU. We call it local prefill when the selected mixed GPU executes prefill and then decode for the same request. The system can instead execute prefill on a prefill-only GPU and transfer the resulting KV state to a mixed GPU for decode. We call this remote prefill.

Suppose a mixed GPU is already executing decode for twenty requests when a new request with a long input arrives. If that GPU executes the new request’s prefill locally, the request avoids a KV transfer and may receive its first output token sooner. The long prefill, however, extends the GPU iterations shared by the twenty active requests and delays their next tokens. With remote prefill, the new request may wait at the prefill-only GPU and then wait for its KV state to be transferred. During that time, the mixed GPU can continue executing decode for its current batch.

Both choices create delay, but they place it on different requests. We call a request already using an affected resource a resident. A useful router must therefore choose both the decoder and the prefill path while accounting for the residents that share those resources.

We evaluated each decoder and prefill-path choice using three user-visible delays. These were time to first token, mean time between output tokens, and end-to-end completion time. A request was good only if it met all three latency objectives. Goodput is the fraction of injected requests that were good. The routing idea was simple. The harder problem was to build an experiment that could test it.

The AI coding agent’s implementation role

We built the prototype on BLIS, the Blackbox Inference Simulator. BLIS is an open-source discrete-event simulator created by our team. It advances a simulated clock from one event to the next. An event can be a request arrival or the completion of a GPU iteration. This structure allowed us to replay the same arrivals under different routing choices and inspect the queue, batch, and completion record for each request. The routing extension and experiment harness are separate from the public BLIS repository.

The agent implemented routing rules inside this prototype. It exposed the score assigned to every possible decoder and prefill-path pair, generated workloads, ran the same arrivals under many routing rules, and checked individual request outcomes. We initially expected to supply the research idea while the agent handled implementation and experiments. That division did not hold. Small implementation choices were changing the meaning of the experiment.

The agent could confirm that every run completed. It could also identify which routing rule produced the highest reported goodput. We still had to ask whether that number measured what we intended. Was the event sequence physically possible? Did the router use only information available at decision time? Did every routing rule receive an equally demanding workload? Each of these questions exposed a problem.

Capacity measurement and disaggregation fraction selection

Before comparing routing methods, we had to decide how fast requests should arrive. The same arrival rate can place very different demands on the same fleet. Interactive requests have moderately long inputs and short outputs. Reasoning requests have shorter inputs and much longer outputs. Deep-research requests have very long inputs and short outputs. We therefore needed a separate capacity measurement for each workload. Our simulated fleet contained one prefill-only instance and two mixed instances that could execute both prefill and decode. The disaggregation fraction describes how the prefill work is divided between them. A fraction of 0 keeps every request’s prefill on its selected mixed instance. A fraction of 1 executes every request’s prefill on the prefill-only instance and then transfers its KV state to a mixed instance for decode. On the H100+A100 fleet, we also varied the decoder assignment fraction, which is the fraction of requests whose decode executes on the A100 mixed instance.

Our first attempt used synthetic request streams and tested only disaggregation fractions 0 and 1. For each fraction, the agent tried several offered arrival rates, measured in req/s, and selected the highest rate for which the reported completion rate was at least 95 percent of the offered rate. The intended physical meaning was simple. If 100 requests arrive during an observation period and about 95 finish during that period, the fleet is approximately keeping pace with the arrivals. If only 80 finish, roughly twenty requests have been added to the queue. If that pattern continues, the queue will keep growing. The agent applied this check to both disaggregation fractions and used the larger result as the capacity of the workload and fleet.

When I reviewed the experiment, I asked what this capacity represented physically. Changing the disaggregation fraction changes where work waits. With a fraction of 0, prefill and decode compete for time on the two mixed instances. With a fraction of 1, prefill work can accumulate at the prefill-only instance, while every request must still execute decode on a mixed instance. Different fractions can therefore produce different limiting queues and different completion rates. That is physically reasonable. However, we must first measure the sustained completion rate of each fraction correctly. Only then can we define the empirical capacity of the workload and fleet as the largest rate observed among the fractions we tested.

The agent traced the existing calculation and found that it did not measure sustained service. Each simulation injected a finite sequence of requests. The reported completion rate divided the total number of completions by the time from the beginning of the run until the final request finished. This interval included startup, random gaps between arrivals, busy service, and the final period in which no new requests arrived and the remaining queues drained. One result made the problem visible. An offered rate of 3.0 req/s failed the 95 percent check, while 3.5 req/s passed. The increase in completion rate was not itself suspicious because a higher arrival rate can fill GPU batches and improve execution efficiency. The problem was the resulting classification. The calculation claimed that the fleet could not keep pace at 3.0 req/s but could keep pace at 3.5 req/s. The pass-or-fail decision did not match the physical meaning that we had assigned to it.

We replaced that check with a direct measurement of sustained service. For each workload and fleet, we supplied enough requests to keep the limiting resource busy. We held the disaggregation fraction unchanged during each run. On the H100+A100 fleet, we also held the decoder assignment fraction unchanged. We counted completions only during the middle 80 percent of the run so that startup and final draining did not determine the result. Each fraction combination was tested with three independently generated sequences of arrival times and request lengths. We averaged the number of requests completed per second across these three runs. The largest average among the tested fraction combinations became the measured capacity for that workload and fleet. It was an empirical maximum over the fractions we tested, not a theoretical hardware limit or a production measurement. On the homogeneous H100 fleet, the measured capacities were 14.4 req/s for interactive traffic, 0.256 req/s for reasoning, and 0.939 req/s for deep-research traffic.

This correction answered how fast the fleet could complete requests, but it did not tell us which fractions would provide the strongest fixed comparison for goodput. The agent initially reused the fractions that had maximized completion rate. I asked why a fraction that maximizes completed requests per second should also maximize the fraction of requests meeting their latency objectives. Following one long-input request makes the difference concrete. If a mixed instance executes its prefill locally, that prefill shares the GPU with requests already executing decode and delays their next output tokens. If the prefill-only instance executes the same work, those existing decode requests can continue, but the new request may wait at the prefill-only instance and for its KV-state transfer. When the fleet is saturated, the main question is which division of work lets the fleet finish the most requests per second. Below capacity, both choices may finish every request, but they can make different requests wait. The better disaggregation fraction for goodput is the one that allows more requests to meet all three latency objectives, even if it does not increase the maximum completion rate.

The final experiment therefore made the two selections separately. We first measured capacity using sustained completion rate. We then offered each workload 60, 80, and 95 percent of that capacity and tested the fractions again. This time, we selected the disaggregation fraction and decoder assignment fraction that produced the highest goodput on request sequences reserved for making this choice. After selecting the fractions, we evaluated every routing method four times for each workload, fleet, and offered load. Each repetition used a newly generated sequence of arrival times and request lengths. Every routing method received the same four sequences, and none of them had been used to select the fractions. This reduced the chance that one unusually easy or difficult request sequence would determine the comparison.

The distinction materially changed the experiment. For interactive traffic on the homogeneous H100 fleet, a disaggregation fraction of 0.4 produced the highest sustained completion rate. The fractions that produced the highest goodput were 1.0, 0.8, and 0.9 at low, medium, and high load. Across all 18 workload, fleet, and load combinations, choosing fractions for goodput rather than capacity raised their mean goodput from 0.9198 to 0.9296. At 95 percent load for homogeneous interactive traffic, goodput rose from 0.902 to 0.944. These fractions still did not form an oracle. They specified how many requests would execute prefill remotely, but not which individual requests would benefit most. They also could not react to cached tokens, live queues, or requests already executing.

Goodput tuning removed most of the apparent advantage over fixed routing. The resident-aware router nevertheless remained the most robust tested routing method that reacted to the current system state. It performed better than the goodput-tuned fractions in 11 of the 18 combinations, tied in three, and performed worse in four. Its benefit was not a universal or dramatic increase in goodput. It was the ability to remain close to the best result across workloads and loads without knowing the best disaggregation fraction in advance. The agent could run both sets of experiments once we defined them. Human judgment was needed to recognize that sustained completion rate and latency-objective attainment were separate scientific questions. The smaller final advantage was more credible precisely because the comparison had become harder.

Output-length leakage in the deployable estimator

Another problem appeared in an admission-delay estimator. We kept an oracle estimator for diagnosis and deliberately allowed it to use the output length that a request would eventually produce. The deployable estimator could use only information available when the router made its decision. During implementation, both estimators received the same diagnostic record, which contained the realized output length. The deployable estimator used that value.

The contaminated estimator produced a median ratio of realized to predicted admission delay of 1.29. A ratio of 1.0 would be exact, while 1.29 meant that the realized delay was 29 percent larger than the prediction. The result passed our first checks. We then asked why the deployable and oracle outputs were identical when their estimated and realized output lengths were different.

The agent followed the value through the sequence of function calls and found the leak. We separated the data passed to the two estimators, prevented the deployable estimator from using the future output length, and withdrew the 1.29 result. The arithmetic had been tested, but the experiment had not enforced the intended information boundary. We had to define that boundary. The agent then checked every function that could pass future information to the deployable estimator.

Overlapping intervals in the remote-prefill estimator

An early estimate for remote prefill added four intervals. They were prefill queueing and execution, KV-state transfer, the decoder queue seen at arrival time, and the first decode iteration. Each interval was real. The problem was that some of them passed at the same time.

Suppose remote prefill and KV-state transfer take 8 ms while 6 ms of the original decoder queue clears. When the KV state reaches the decoder, only 8 ms has passed. The old estimate reported 14 ms because it added both intervals in full. This double counting made remote prefill look more expensive than it was. Only the part of the decoder wait that remains after the transfer can delay admission.

We asked which event had to finish before the next event could begin and which events could overlap. The agent replayed their timestamps and showed where the intervals overlapped. Our first repair removed the double count. The final estimator advanced the scheduler state one simulated iteration at a time until the request was admitted and produced its first token. More samples could not repair the old formula because we had to correct the order of events.

The human, agent, and simulator workflow

We did not begin with a complete specification for the agent to execute. Results from the implementation often exposed a contradiction. We used the contradiction to state the scientific requirement more clearly and ran the experiment again. The investigation usually followed four steps.

1. We stated the required physical behavior, the information available when the router acted, and the request sequence that every routing rule had to receive.
2. The agent made that requirement observable. It traced a metric to the code that computed it, exposed hidden state, replayed events, or tested values near a pass-or-fail threshold.
3. BLIS produced each request’s history and reran the same arrival while forcing a different decoder or prefill path.
4. We changed the implementation, protocol, or claim to match the measured record. Then we repeated the process.

The simulator did not choose the research question, and the agent did not decide what the experiment should mean. Those decisions remained ours. We compared the simulator record with the physical requirements we had stated. The system had the final word on the implemented experiment because its record showed whether the implementation supported the claim.

Strengths and limitations of the AI coding agent

The agent worked best when a result could be checked against a concrete record. It instrumented every possible decoder and prefill-path pair, reran an arrival with local and remote prefill, gave each routing rule the same request arrival times and token counts, verified that each request reached exactly one outcome, and located the code that produced a number. It also reduced the cost of being wrong. We could test an explanation and discard it without treating the implementation effort as a reason to defend it.

It was less reliable when correctness depended on the meaning of the model. The agent could test a calculation without recognizing that the calculation described the wrong physical process. The program could be correct while the scientific claim was not.

The agent usually accepted the quantity or relationship named in the task. If we described queueing delay as a sum, it implemented and tested that sum. If an experiment script labelled an arrival rate as the largest sustainable rate, it analyzed results at that rate. Progress came when we asked what the calculation represented in the physical system. The agent then answered each question with code, event records, and repeatable experiments. This combination let us inspect far more evidence than we could have handled manually.

Corrected benchmark design and results

After these corrections, the final benchmark covered three workloads and two fleets at offered rates equal to 60, 80, and 95 percent of measured capacity. This produced 18 workload, fleet, and offered-rate combinations. We tested three routing rules that chose a decoder and a prefill path for every arriving request while observing the current system state. We also tested two ways of keeping the disaggregation fraction and decoder assignment fraction unchanged during a run. One used the fractions that had produced the highest sustained completion rate. The other used the fractions that had produced the highest goodput on separate request sequences reserved for making that choice. All five routing rules then ran on four new request sequences that had not been used to choose rates or fractions. This produced 360 runs. Every submitted request completed, and none was dropped, timed out, or truncated.

Across the 18 workload, fleet, and offered-rate combinations, the resident-aware router achieved a mean goodput of 0.9327 when every combination received equal weight. For each combination, we averaged each of the three routing rules that observed current system state over the four new request sequences. We then compared the resident-aware router with the largest of those three averages for that same combination. This did not select one routing rule after viewing the full benchmark. The resident-aware router’s largest shortfall was 0.0125, which corresponds to 1.25 fewer good requests per 100 arrivals in that combination. Its mean goodput was 0.0031 higher than the result obtained with goodput-tuned fixed fractions.

The corrected comparison changed the conclusion. The resident-aware router still ranked first among the three routing rules that observed current system state, but its advantage over the goodput-tuned fixed fractions was small. The evidence supported a claim about robustness across the tested conditions, not a universal win.

Methodology for AI-assisted systems research

Our experience suggests a practical role for AI in systems research. For every scientific claim, ask the agent to build a trace, replay, test, or implementation that could prove the claim wrong.

We now ask four questions about every major result. What physical event or resource does the metric represent? What information is available when the system acts? Do the load and controls produce a fair comparison? What can the evidence establish, and what would make us withdraw the claim?

The human researcher remains responsible for these questions because they define the object being studied. The agent can make the answers visible in traces, invariants, matched replays, and repeatable experiments. The system then supplies the record used to judge the claim.

Lessons from the study

The agent made it easy to run experiments, which made the quality of each question more important. A vague question could produce hundreds of completed experiments before we noticed the ambiguity. A precise question could be tested at a depth that had previously been impractical. We learned to use each part of the collaboration for a different purpose. We challenged the meaning of a result, the agent challenged the implementation through traces and replays, and the simulator showed whether the implementation behaved as required.

The routing rule was one outcome of this project, but the broader contribution was an AI-native research method. The agent did not replace scientific judgment. It reduced the cost of turning a question into instrumented code, running controlled comparisons, and tracing a surprising result back to the events that produced it.

The central lesson is to use that speed to make claims easier to challenge, not merely faster to produce. The researcher must define what a metric means physically, what information the system may use, and what makes a comparison fair. The agent can turn those decisions into tests and expose where the implementation violates them. The researcher frames the question, the agent makes it testable, and the system record has the final word.

About the author

Vishakha Ramani is a Research Staff Member at the IBM T. J. Watson Research Center, where she works at the intersection of theory and systems, with a focus on large-scale LLM inference and cloud platforms. She received her M.S. and Ph.D. degrees in Electrical and Computer Engineering from Rutgers University in 2020 and 2024, respectively, and conducted her graduate research at WINLAB. Her research applies performance modeling, queueing theory, and systems experimentation to improve latency, throughput, and information freshness in distributed and AI systems.

Disclaimer: These posts are written by individual contributors to share their thoughts on the Computer Architecture Today blog for the benefit of the community. Any views or opinions represented in this blog are personal, belong solely to the blog author and do not represent those of ACM SIGARCH or its parent organization, ACM.

This is the second (and final) post in the series covering the Next Generation Branch Prediction Champtionship (CBP-NG). Part-I details the provided infrastructure and the winning submissions.

Distilled Branch Predictors (Simha Sethumadhavan – Columbia University)

The author proposes an ‘Ensemble Predictor’ composed of a fast, one-cycle predictor P1 and a slower (but more accurate) two-cycle predictor P2. However, instead of training them both independently on the actual branch outcomes, P1 is trained on the predictions made by P2 (which continues to be trained on branch outcomes), turning it into a faster, distilled version of the more accurate predictor. This student-teacher model improves the VFS score by lowering the misprediction recovery latency. An additional PC-tagged structure is added to identify one-sided branches and prevent the usage of P2 to predict them, further lowering the energy consumption. In the final submission, a bimodal predictor is used as P1, and P2 is an 8-table TAGE predictor.

Run-Ahead Block TAGE (Prakhar Gupta, Yuwei Sun, Rishav Sen, Reet Sinha, Swetha Karthikeyan – University of Illinois Urbana-Champaign)

The authors use a single (15-table, geometric history length) TAGE predictor. Ahead Pipelining is implemented to hide the table access latency, so that the effective prediction latency is one cycle. To correctly use ahead-pipelining, a secondary tag is added to all table entries; an entry can be used only if the tag calculated from the last branch PC of the previous block matches with the stored tag. The resulting increase in throughput outweighs the increase in TAGE MPKI, boosting the VFS score. A maximum of 7 branches can be predicted at once, over a 256-instruction predict block. An 8k-entry bimodal predictor is used as a fallback when TAGE is unable to provide a prediction. Independent Resolution Chains allocate a tag, usefulness and hysteresis field to each slot in the predict block, allowing each slot to use a different TAGE table. Using independent history length per branch improves the MPKI for most cases, but is detrimental when there is intra-block correlation.

Coding Agents as Design Searchers: An Autonomous TAGE Tuning Campaign
(Matt Pallan – University of Maryland)

This submission stands out because its primary contribution is the methodology, not the predictor. The author proposes a systematic approach to leverage AI for design space exploration – the tasks are split between two independent agents Ag1 (Claude Code) and Ag2 (OpenAI Codex). Ag1 is responsible for coming up with structural (table size, hashing, indexing, etc.) or behavioral (update policy, initial value, etc.) changes and their implementation. Ag2 acts as a critic – questioning the intent behind Ag1’s changes and coming up with out-of-the-box architectural ideas. The agents stay in sync through shared collateral – a journal and a machine-readable ledger, which are updated after every experiment. The only human inputs for this study are the objective (maximizing the VFS score), the record-keeping mechanism and some hard constraints to reduce the search space (such as capping the predictor latency at 2 cycles). Each configuration proposed by Ag1 is first tested on a reduced instruction count on the provided traces, only those that perform well (31 out of 2300+) are run on the full instruction count. Out of these, only six changes proved beneficial and survived cross-inspection by Ag2. The final TAGE model is tuned to prioritize throughput over accuracy and ditches the Statistical Corrector mechanism as it is deemed infeasible for HARCOM implementation.

Offset-Free TAGE-SC
(Yashwant Kumar Balivada, Sairam Viswanathan Susarla – Texas A&M University)

A two-level predictor is implemented – the first level is a single-cycle gShare that gives fast predictions. The second level consists of a TAGE-SC predictor which is more accurate. The key distinction of the latter is its usage of Branch Number instead of Branch Offset within a predict block. This leverages the championship-specific assumption that we can magically identify conditional branches, to reduce storage and energy expenditure. The VFS score is maximized when predicting upto 8 branches in a 64-instruction window. The Statistical Corrector uses 3 components –  branch-PC bias, taken branch history and global path history.

Key Takeaways

The usage of  TAGE was a common trend in the championship, with all the submissions implementing some form of tagged history tables in their predictor to reduce MPKI. Ahead Pipelining was another popular technique that many of the contestants used to hide the table-access latency for large predictors. This was also the first championship to have a submission developed by AI agents; the easily reproducible methodology holds great potential for future computer architecture research.

This was the first Branch Prediction Championship to make energy consumption a part of the final score. Naturally, the submissions saw a shift from the highly accurate and complex predictors that dominated the past championships to simpler, implementation-friendly models. This, along with the usage of HARCOM to model the predictors in a hardware description language (or something close to it) ensured that the industry can easily adopt the novel ideas that were proposed.

However, despite this imposition of practicality, the provided infrastructure made some simplifying assumptions that were criticized by some of the popular names in Branch Prediction. The first issue was the magical detection of conditional branches by the simulator, which is quite unrealistic. Allowing the contestants to implement a BTB (Branch Target Buffer) and RAS (Return Address Stack) could help paint a more complete picture. Secondly, the usage of high throughput techniques like Ahead Pipelining requires some consideration of the misprediction recovery mechanism, which is a key stage in any high performance branch predictor.

About the Author

Digvijay Singh obtained his Bachelor’s degree from BITS Pilani, India and his Master’s degree from Texas A&M University where he worked on data prefetching as part of the CAMSIN research group. He currently works as a Silicon Architect in Google’s mobile CPU team.

Disclaimer: These posts are written by individual contributors to share their thoughts on the Computer Architecture Today blog for the benefit of the community. Any views or opinions represented in this blog are personal, belong solely to the blog author and do not represent those of ACM SIGARCH or its parent organization, ACM.

Background

The Next Generation Branch Prediction Championship (CBP-NG) aimed to address practical implementation challenges in branch prediction, moving beyond metrics used in earlier iterations. By emphasizing energy-adjusted performance—incorporating throughput, timing, and dynamic energy alongside traditional Misprediction Per Kilo-Instruction (MPKI) metrics—the championship sought to align research with industry design constraints. Utilizing an extensive set of 1579 traces derived from ARM and Ampere architectures, the event evaluated predictors through a two-level model: a first-level (P1) predictor for immediate output and a second-level (P2) predictor that verifies and corrects potential mis-predictions. All submissions were required to be implemented using the HARCOM C++ library that imposes restrictions on the behaviour of all variables and structures, treating them as hardware components such as wires, registers and SRAMs arrays. The energy consumption was approximated based on the ‘hardware’ used, assuming 5nm FinFET technology. The final scoring was done using a mathematical formula that combines:

• IPC: Number of instructions predicted per cycle
• CPI: Number of mispredictions (by the L2 predictor) per correct-path instruction times misprediction penalty
• EPI: Energy consumed per correct-path instruction

into a Voltage Frequency-scaled Speedup (VFS) score. The objective is to maximize this VFS score.

In this blog post, we first describe the gSHare-N-ahead predictor, which delivers the highest VFS score among all example predictors provided with the infrastructure. It implements a mechanism called ahead-pipelining, which inspired several submissions in this championship. This is followed by an overview of the championship winners – starting from first place to third. The remaining submissions will be discussed in Part 2 of this blog post.

GShare N-Ahead Predictor

GShare N-ahead is one of the predictors provided as part of the simulation infrastructure. It supplements the conventional N-wide (N predictions per cycle) gShare predictor with ‘Ahead Pipelining’. Instead of using the current block B1’s address (PC_B1), it uses the prior block B0’s address (PC_B0) to index the prediction tables. Since PC_B0 is available one cycle earlier than PC_B1, predictions can be made earlier, effectively reducing prediction latency. The only catch is that in a predict block with N branches, there can be N+1 paths leading out of it, so until PC_B1 is known, we have to consider all possibilities and make (N+1)*N predictions in BP0. Once PC_B1 is known in BP1, it is combined with B0’s last conditional branch direction and the number of conditional branches in B0 to multiplex the BP0 predictions to obtain the final N predictions.
The particular implementation used in the championship predicts N = 7 branches per cycle as a P1 predictor; P2 simply reuses the P1 prediction for super-aggressive prediction throughput.

MORSL: Minimal-Overhead Rank-Based Predictor with Summation-Free Correction and Lazy Access (Toru Koizumi, Masanari Mizuno, Soma Nishida – Nagoya Institute of Technology; Kanata Abe – The University of Tokyo; Tomoaki Tsumura – Nagoya Institute of Technology; Ryota Shioya – The University of Tokyo). First place

The authors started with a TAGE predictor for high prediction accuracy, and implemented the following features: 

• Ahead-pipelining is enabled for a higher throughput.
• While deciding the TAGE history lengths, the shortest history length is increased to compensate for the reduced context due to ahead pipelining. Furthermore, the larger history lengths are spread farther apart than the smaller ones. This allows for pseudo-skewed associativity within smaller tables, since each TAGE table has a single tag per entry. 
• A Tagged Corrector (TC) mechanism consisting of two tagged predictors.  TC-Bias is indexed using the current block address and can override the TAGE prediction  in case of a high confidence tag match. TC-History uses the previous block’s address, BrIMLI and Path History to override the TAGE + TC-Bias prediction in case of a tag match. Using overrides instead of summation reduces energy consumption by avoiding multiple table reads on the adder datapath.
• An ‘allocation guided’ access filter is implemented (as two 1-bit tables indexed by PC and PC XOR shortest history), which records whether a branch PC had any allocations in a longer TAGE table in the past. The intuition is that if the PC did not trigger allocations to a longer table, the shorter tables were able to accurately make predictions for it. Unless both the access filter table reads return 1, the ‘mini-TAGE’ mode is activated and only the two shortest TAGE tables are read to save energy.

Ahead-Pipelined N-branch GShare with Tagged Tables (Jun Fan). Second Place

This proposal improves the accuracy of the ahead-pipelined gShare predictor by using 18-bit history and 2 hysteresis bits. Furthermore, the gShare RAM size is halved to make room for two ahead-pipelined tagged tables (20 and 80-bit histories, respectively). These three tables are looked up in parallel during prediction, and the longest matching entry is used to make the first stage prediction P1. The final prediction P2 simply reuses P1 to maximize throughput. Upon a branch misprediction, the table that produced the prediction is updated, and new entries are allocated in tables with longer histories.  If a wrong prediction came from the 20-bit table, a new entry will be allocated only in the 80-bit table, but if it came from the 18-bit table, the 20 and 80-bit tables will both have new allocations. Finally, a direct-mapped ‘bias’ table is added which stores a PC along with  a 3-bit saturating counter in each entry. When gShare is used to make the final prediction and the corresponding PC is found in the table with a strong bias (000 or 111), the predictor is updated with the stored strongly biased value.  

Energy-Efficient Ahead-Pipelined TAGE
(Nhat Dang, Eric Rotenberg – North Carolina State University). Third Place 

The authors implement a single branch predictor to reduce the energy consumption due to multiple table reads and updates in a dual-predictor scenario. Ahead Pipelined TAGE is implemented, consisting of a baseline bimodal predictor and 10 tagged tables with geometric history lengths. It uses the previous block’s address to predict upto 4 branches per cycle in a 256-instruction region. A 6-bit secondary tag (generated from the lower bits of the last branch PC) is added to the TAGE entries to allow disambiguation of the missing history for the previous block. Additionally, the access to  6 largest TAGE tables is gated. Specifically, if there have been no mispredictions in the last 512 blocks, and the total number of mispredictions is less than the (number of retired branches / 1024), these tables are not accessed, to save dynamic energy. Further energy optimizations include placing the tag comparison logic near the tag SRAMs, and placing the SRAMs accessed during prediction together.

Stay tuned for Part-2 of this article to learn more about other submissions and the industry/academia’s perception of this contest.

About the Author

Digvijay Singh obtained his Bachelor’s degree from BITS Pilani, India and his Master’s degree from Texas A&M University where he worked on data prefetching as part of the CAMSIN research group. He currently works as a Silicon Architect in Google’s mobile CPU team.

Disclaimer: These posts are written by individual contributors to share their thoughts on the Computer Architecture Today blog for the benefit of the community. Any views or opinions represented in this blog are personal, belong solely to the blog author and do not represent those of ACM SIGARCH or its parent organization, ACM.

Zero-Knowledge Proofs

As conversations about LLMs continue to dominate computer science, the cryptographic community has been abuzz around a different three letter acronym: ZKP. Zero-Knowledge Proofs (ZKPs) allow a prover to convince a verifier that a statement is true without revealing the secret information that makes it true. In other words, ZKPs make it possible to verify computation without exposing private data behind it. 

In a cloud computing context, ZKPs could allow a service provider to convince a client that they executed a service faithfully, without having to reveal their sensitive trade secrets, IP, or (for AI services) model weights. On the other hand, ZKPs could allow an edge user concerned about security or anonymity to prove identification, financial eligibility, or authenticity of image transformation to untrusting services. 

Two flavors of ZKPs have emerged with particularly intriguing tradeoffs: zkSNARKs  (zero-knowledge Succinct Non-Interactive ARguments of Knowledge) and zkSTARKs (zero-knowledge Scalable Transparent ARguments of Knowledge). zkSNARKs are attractive because they generate compact proofs (as small as 192 bytes!) which are fast to transmit and verify. This made them especially appealing for early blockchain adopters of ZKPs. zkSTARKs, on the other hand, offer both faster proving times and post-quantum security, but generate much larger proofs (upwards of several megabytes). This increases communication costs and requires more processing by the verifier. As such, zkSTARKs may be more advantageous in applications where proof size constraints are more relaxed or where the number of verifiers is small.

Both zkSNARKs and zkSTARKs suffer from the computational cost of proof generation. The proving time to generate a zkSNARK for a target application is roughly 5-6 orders of magnitude more than the time to run the original application itself. zkSTARK provers are similarly expensive. Further, as the problem complexity grows (e.g., running an AES is much simpler than running a GPT inference), the corresponding ZKP’s computational cost also grows. Consequently, many otherwise compelling and exciting ZKP applications remain out of reach today. To address this, researchers have begun accelerating both zkSNARKs and zkSTARKs with GPUs, FPGAs, and ASICs. 

Dissecting a ZKP

Modern ZKPs are diverse and defined by combining a polynomial interactive oracle proof (PIOP), which reduces program correctness to polynomial checks, with a polynomial commitment scheme (PCS), which binds the prover to the relevant input polynomials and lets the verifier check claimed polynomial values at specific points. PIOPs and PCSs can be mixed and matched to define many ZKP protocols with different tradeoffs in proof latency, proof size, verifier latency, setup assumptions, and post-quantum security.

The diversity between PIOP and PCS schemes typically results in ZKPs built upon some kernels with extremely high arithmetic intensity on wide bitwidths and others with low arithmetic intensity and complex data movement patterns. Any hardware seeking to accelerate ZKPs must account for both flavors of computational kernel. 

The high data volume follows from how PIOPs represent the execution of a function. Many PIOPs encode a program’s execution trace as one or more polynomials (represented as long vectors of polynomial evaluations). Verifying program correctness then reduces to checking that these polynomials are consistent with the program’s constraints and satisfy a number of algebraic identities. Polynomials are especially useful because, rather than checking these constraints individually across the entire execution trace, they can be encoded as polynomial identities that the verifier tests at a small number of randomly chosen points. These long vectors can range from 2¹⁴–2³⁰ elements, amounting to 0.5 MB – 32 GB for a single such vector. These vectors may be repeatedly transformed, permuted, hashed, or combined, creating substantial pressure on memory capacity, bandwidth, and data movement. 

While PIOPs often rely on more data movement with lower arithmetic intensity, PCSs often behave quite differently. The PCSs found in zkSNARKs primarily rely on modular arithmetic over large prime fields and elliptic curve (EC) groups. Generally, field elements are on the order of 256 bits wide, with some ECs’ field elements being upwards of 384 bits. EC computations are even more computationally expensive than large-bitwidth finite field computation, with the addition of two EC group elements requiring 16 (or more) modular multiplications. zkSNARK PCSs are dominated by the latency to calculate many EC group additions. 

zkSTARK PCSs differ significantly from zkSNARKs. zkSTARK PCSs are built upon repeated hashing as opposed to EC arithmetic. Some zkSTARK PCSs use smaller, hardware-friendly finite fields, which can reduce arithmetic bit widths; others still operate over wide fields on the order of 256 bits. In either case, zkSTARK provers must perform extensive hashing and repeatedly process, reorganize, and extend large vectors of field elements, creating substantial memory-bandwidth and data-movement demands.

Due to differences between PCS and PIOP kernels, accelerating proof generation requires more than support for wide datatypes and long vectors. A practical architecture must efficiently handle both compute-intensive kernels, such as elliptic-curve arithmetic and hashing, and bandwidth-intensive kernels, such as polynomial transform and large-scale data rearrangement.

Accelerating ZKP Kernels

Seeing the promise and current computational limitations of ZKPs, many architects have begun work accelerating ZKPs on GPUs, FPGAs, and ASICs. This largely amounts to accelerating individual ZKP kernels.

The Multi-Scalar Multiplication (MSM) kernel serves as the bedrock of zkSNARK PCSs. MSMs compute the sum of many EC point multiplications, which can be thought of as dot products between vectors of scalar integers (derived from the polynomials) and vectors of two-dimensional EC points (derived from the protocol setup phase). However, scalar multiplication by an EC point is actually achieved via repeated EC point addition (or PADDs) described above. Naively, assuming 256-bit scalars, a single scalar multiplication requires 2²⁵⁶ PADDs. This cost can be tempered by employing a binary representation of scalars and performing a “shift-and-add” algorithm for multiplication, resulting in at most 255 PADDs and 255 PDBLs (point doubles) per scalar that must be computed serially. To improve upon this, most MSM implementations today use Pippenger’s algorithm, a technique that parallelizes computations across independent slices of the scalars, to further reduce the number of PADDs. However, Pippenger’s algorithm is data-dependent upon the scalars being committed. Many MSM workloads are sparse, meaning they contain a disproportionate number of zeros, ones, and other small scalars, while others are dense, consisting of randomly distributed full-width field elements. These differences can significantly change the amount of useful EC work performed, the load balance across parallel workers, and the memory-access behavior. Recent ASIC approaches have proposed dynamically scheduled microarchitectures, contention-free multi-PE datapaths, priority-based bucket scheduling, and reconfigurable modular arithmetic arrays. Recent GPU approaches have proposed fine-grained, load-balanced task mapping, sparse-matrix-based bucket accumulation, adaptive preprocessing for improved time–space trade-offs, and tensor-core-assisted arithmetic and multi-GPU orchestration.

Merkle Trees are the zkSTARK analog to MSMs, enabling commitments to a vector via hashing as opposed to EC cryptography. Constructing a Merkle Tree is equivalent to constructing a binary tree where the vector values being committed are the leaves, and parents are constructed by performing a hash seeded by its two children. The root node of the resulting Merkle Tree is a commitment to the polynomial. Constructing a Merkle tree requires a large number of hash operations: for a tree with N leaves, the prover must compute approximately N-1 internal-node hashes. However, performance is shaped not only by hash throughput, but also by how the tree is traversed. A conventional breadth-first construction materializes each tree level before computing the next, creating substantial intermediate-memory traffic. Recent ASIC approaches like the MTU use a hybrid traversal that preserves parallel execution while retaining partial reductions locally, reducing the need to repeatedly write and reread intermediate tree nodes from memory. Other approaches tile Merkle trees into scratchpad-resident subtrees and process each subtree entirely on-chip using spatial arrays, while GPU approaches pipeline batched Merkle trees across layer-specific GPU kernels to sustain thread utilization while overlapping data transfers with hashing. Recent work also explores FPGA acceleration of ZK-friendly hash functions to increase hash throughput. 

Number Theoretic Transforms (NTTs) are analogs of the FFT but over large, ~256 bit finite field elements. NTTs are primarily used to reduce the cost of polynomial multiplication from O(N²) operations to O(N log N). The key challenge for NTTs is handling variable memory access patterns in between NTT stages and handling off-chip memory transfers when computing NTTs whose working set doesn’t fit into on-chip caches and scratchpads. Recent approaches have focused on recursive decompositions, pipelined MDC based architectures, and constant geometry approaches.

SumChecks are another major prover kernel for many PIOPs. At a high level, a SumCheck prover repeatedly streams large tables of polynomial evaluations, performs elementwise finite-field computations determined by the polynomial being evaluated, and reduces the results into progressively smaller tables. Although each round is built from additions, multiplications, and linear combinations, these operations are performed over wide field elements and must be applied across large vectors, creating substantial arithmetic and memory-bandwidth demand. While this seems relatively simple, unlike NTTs, SumCheck’s dataflow changes with the polynomial. More input polynomials or higher-degree expressions require more elementwise evaluation products, increasing both the compute load and memory pressure for storing  intermediates during folding. Initial work addressed this with vector architectures and fixed-function ASICs to exploit short-term data reuse within streaming frameworks. Recent work like zkPHIRE has proposed programmable ASICs to efficiently handle variable dataflow, decomposing SumCheck into common primitives—table updates, extensions, and evaluation products—and mapping them onto programmable compute pipelines, allowing the support of arbitrary polynomial structures and gate types.

The above kernels typically account for the vast majority of ZKP compute time, but all ZKP protocols also rely on a variety of smaller kernels to stitch these major components together. In order to avoid Amdahl’s Law bottlenecks, it is essential ZKP hardware accelerates these kernels as well.   

The Future of ZKPs

ASIC acceleration of ZKPs has evolved from fixed-function modules towards reconfigurable and programmable datapaths. Most existing accelerators remain specialized to one or a small number of protocols, while recent work has begun to support multiple protocols. Recent characterization studies on CPUs and GPUs show that execution bottlenecks, memory behavior, and scalability vary substantially across protocols and implementations. As ZKP protocols continue to evolve, there is growing interest in more general architectures that can accommodate its heterogeneous kernels, arithmetic requirements, and execution patterns. Understanding how different protocols stress the hardware stack remains an active area of research. 

To learn more and join the conversation with architects working to make real-time, deployable ZKPs a reality, join us at the ZKARCH workshop at MICRO 2026!

About the Authors:

Alhad Daftardar is a Ph.D. candidate in Electrical and Computer Engineering at NYU. He received his BSEE from Georgia Tech and his MSECE in VLSI from the University of Michigan. His research interests are in computer architecture for Zero-Knowledge Proofs and privacy-preserving computation.

Brendan Sweezy is a Ph.D. student at New York University studying Electrical and Computer Engineering. He received his BS from Duke University. His research focuses on hardware acceleration for Zero-Knowledge Proofs, particularly targeting programmable and edge-scale devices.

Brandon Reagen is an Assistant Professor of Electrical and Computer Engineering at New York University with a focus on computer architecture. His work has been recognized with the ASPLOS’25 Best Paper Award, multiple best paper nominations (DAC, PACT, HASP), and multiple Top Pick/honorable mention awards. He has been a performer (and PI) on the DARPA DPRIVE, PROWESS, and COOP programs, was recognized as a DARPA Riser, and won the NSF CAREER award. He is a lead PI on the NSF CIRC Grand Cryptolets project and has generously received support from Google, DTCC, and AMD.

Disclaimer: These posts are written by individual contributors to share their thoughts on the Computer Architecture Today blog for the benefit of the community. Any views or opinions represented in this blog are personal, belong solely to the blog author and do not represent those of ACM SIGARCH or its parent organization, ACM.

LLM execution requires moving gigabytes of data, which includes static model weights, the generated KV cache, and other intermediate representations. Depending on the deployment, this may involve communication across the network, PCIe links, accelerator links (e.g., NVLink), and memory interconnects, often leaving LLM execution bottlenecked by the data movement feeding it.

An obvious mitigation for this overhead is to simply reduce the amount of data transferred. Quantization is a field of research that adopts this principle, where tensors are shrunk to a smaller data format, reducing the space and interconnect traffic needed for inference. There’s a rich body of work examining quantization’s usefulness on different LLM components like weights, activations, attention, and the KV cache.

While quantization has earned its place in the field, it is difficult to universally deploy. Model accuracy degrades due to loss of information, requiring quantization to be tuned for specific models and use cases of LLMs. It has been shown that even when model accuracy appears to be unaffected, side effects manifest in unpredictable ways, such as increased rambling from quantized LLMs. The impact of quantization varies considerably across model architectures and workloads, making it difficult to guarantee consistency after deployment.

Lossless compression: Reducing data without losing information

Is it possible to reduce data volume without compromising on accuracy? Lossless LLM compression is an emerging line of work that demonstrates this. All information is retained, preserving model accuracy, while data volume is also reduced. While necessarily providing less compression than quantization, the reduction in volume is significant (1.4x average data reduction).

Lossless compression is not without challenges. Tensors have to be reconstructed, leading decompression to lie along the critical path, potentially overriding the benefits of reduced data transfers. The floating point values contained in tensors can also be difficult to compress. For example, outlier values must be preserved, requiring additional tracking metadata.

Fortunately, structure within LLM tensors provides compression opportunities. Models typically use BF16 data type, containing 1 sign bit, 8 exponent bits, and 7 mantissa bits (Figure). The values of the mantissa bits follow a fairly random distribution, but the exponent bits exhibit structure. While 8 bits are provided, usually only ~3 bits carry useful information. The other 5 bits are low entropy. 

Figure. Exponent bits for BF16 LLMs are highly compressible. Figure reproduced from DFloat11 paper with authors’ permission.

A few lossless compression schemes have emerged that take advantage of this structure, demonstrating improvements for LLM inference. These schemes are developed for GPUs, as they are typically the processor of choice for LLM execution due to their high throughput. The mechanisms described here partition the compressed stream into independent units so that thousands of GPU threads decompress in parallel, generating significant decompression throughput. The schemes minimize metadata to promote GPU cacheability and avoid extraneous overhead of metadata accesses during decompression. 

For example, DFloat11 targets cases where LLM weights exceed GPU memory capacity. Excess weight tensors are kept in CPU memory and transferred across PCIe as needed, this is known as offloading. The key idea is to use offline Huffman coding to compress the exponents ofBF16 weights, replacing these 8 bits with fewer bits that act as lookup keys. Compact lookup tables are generated that fit within the GPU’s shared memory. Leveraging DFloat11 lossless compression, model sizes are reduced by ~30%, allowing a 48 GB model to fit in a 40 GB GPU. This translates to 2 – 40x faster inference over HuggingFace Transformers, as data is no longer moved across PCIe.

Going further, ZipServ investigates data movement that occurs within the GPU, specifically, when bringing model weights from the GPU memory into tensor cores before performing matrix multiplication. Similar to DFloat11, ZipServ replaces the 8 exponent bits of BF16 LLM weights with 3 bits, compressing the top 7 most frequent exponent values offline. The remaining exponents map to a special value, which indicates the exponents are stored in full form in a separate location. Fused matrix multiplication kernels fetch and decompress directly into tensor cores, avoiding the memory round trip typically required for decompression. This increases throughput by 22% on average over vLLM.

Our work, Invariant Bit Packing (IBP), generalizes compression and makes it data-type agnostic. It enables not only runtime decompression, but also compression, extending the use case to include the runtime-generated KV cache. IBP discovers invariant bits, i.e., repeated low-entropy bit values across tensors, and stores these bits as metadata in the GPU, small enough to fit in GPU shared memory. Identifying invariant bits across a small subset of tensors is sufficient to generalize to the full set. When compressing, these bits are removed from the tensors, while decompression adds the invariant bits back using cheap bit shift operations. IBP achieves an average of 24% faster LLM inference over FlexGen and InfiniGen, by decompressing tensors on-the-fly while transferring from the CPU. This improves both weight and KV cache transfers, demonstrating the generality of lossless compression.

This topic is making its way to deployment. Cloudflare revealed Unweight for NVIDIA H100 lossless decompression, building on the ideas previously discussed. IBM Research demonstrated ZipNN boasting similar lossless compressibility for LLMs, targeting storage and network transfers. Doubleword.ai has shown that newer FP8 models are also compressible due to the same exponent redundancy.

The next frontier: Compressing the full data path

Lossless compression is having its moment for GPU data movement optimization, but the GPU is not the only place where compression can help. In scale-out LLM deployments, model and computations are split across multiple axes (data, tensor, sequence, pipeline, and expert) referred to as 5D parallelism. Each axis can add new data traffic. For example, tensor parallelism adds partial activation movement, incurring overheads of 20% on NVLink transfers in multi-GPU deployments. Deploying across multiple machines exacerbates this problem, as these rely on lower bandwidth network connections. Further, expert parallelism, where specific LLM layers are split across GPUs, requires all-to-all communication to move intermediate tokens, incurring 40% overhead. All of this movement could be compressed.

A fundamental question is where the decompression should happen. Using the GPU for decompression can steal cycles from other concurrent compute tasks. Conversely, if we decompress early, e.g., at a network smart switch or SmartNIC, the data grows before crossing PCIe or NVLink, losing benefits for downstream links. Depending on the deployment, either approach may have acceptable tradeoffs. NVIDIA B200 GPUs demonstrate decompression along the transfer path by including a new hardware decompression engine that transfers compressed data across PCIe and then decompresses without utilizing GPU multiprocessors. It currently supports a limited range of algorithms (LZ4/Snappy/Deflate). How to extend this range and make the engine flexible, is an open question.

Hardware architects have other potential substrates for decompression present within the memory system: processing in or near memory can store data in compressed form in memory, then decompress data as it leaves memory, increasing effective GPU memory capacity without reserving GPU cores for decompression. This allows more data to be kept within GPU memory, conversely reducing the volume of data needed to be transferred across low bandwidth links like PCIe.

Adapting lossless compression for dynamically changing data is another significant open problem. All the schemes we discussed target static data. This works well for inference, where data values (e.g., KV cache entries or weight tensors) are reused across iterations, amortizing compression overhead. Training breaks this assumption. The entire working set changes every iteration, needing repeated loops of compression and decompression. Mitigating this overhead requires lossless compression that can quickly recompress data. More ambitiously, a scheme that supports tensor updates (e.g., gradient descent) on compressed data could eschew the need for recompression entirely.

Conclusion

Lossless compression for LLMs is still in its infancy, but already showing promise. Existing work has demonstrated that LLM tensors can be compressed by ~30%. The overhead of communication in multi-GPU deployments is significant (20 – 40%), providing a prime future use case. However, it requires tailoring compression to the intricacies of the intervening links, and involves identifying the best candidate to perform decompression. This is an open research field ripe for investigation.

About the Authors:

Aditya K Kamath was a Ph.D. student in Computer Science and Engineering at the University of Washington (UW). His work revolved around analyzing data movement and mitigating its performance impact in memory-intensive applications.

Simon Peter is an associate professor in Computer Science and Engineering at UW. His work is in low latency, scalable, and energy-efficient data center systems and cloud application design, across the hardware/software continuum. 

Disclaimer: These posts are written by individual contributors to share their thoughts on the Computer Architecture Today blog for the benefit of the community. Any views or opinions represented in this blog are personal, belong solely to the blog author and do not represent those of ACM SIGARCH or its parent organization, ACM.

AI is starting to shape architectural mechanisms, workloads, and evaluation. To make sense of it, we need a compact, shared way to preserve enough of that process for other groups to evaluate and build on AI-assisted claims.

At the 53rd ISCA in Raleigh, AI for architecture stopped feeling like a side conversation. In the hallways, the talk kept coming back to one thing. AI is starting to enter the architecture design loop, the repeated process of framing a problem, proposing or editing a design, measuring it, rejecting weak candidates, and deciding what to try next.

In particular, there were two deep-dive workshops coupled with other activities. The MLArchSys workshop added A³, a segment on agentic approaches to architecture, and the full-day Architecture 2.0 workshop focused entirely on agentic design. Both drew well over a hundred people and were standing-room-only by the end. The same shift was visible in the main program, in a plenary panel on research and education in the GenAI era.

The Architecture 2.0 workshops at ISCA 2026.

The MLArchSys workshops at ISCA 2026.

Full rooms on a particular subject matter are undoubtedly a sign of community momentum. The opportunity now is to turn that momentum into a durable engineering practice. That will take shared evidence, reusable tools, and enough agreement for a claim to leave the room where it was born and still be checked, compared, taught, improved, or rejected by someone else. AI is already producing architectural ideas. The question is what must travel with those ideas for them to become engineering knowledge.

Suppose a paper reports an AI-generated memory prefetcher with a 15 percent speedup. The code runs, and the speedup reproduces under the reported setup. But the agent saw some workloads and not others, adapted to simulator feedback, tried many candidates, and picked this one. What, exactly, is the contribution here? The final prefetcher? The speedup? The prompt? The agent? Or the process that connected them?

Once AI chooses workloads, responds to feedback, and selects which candidate to report, the same uncertainty about what counts as the contribution reappears for every such result. As AI gains more influence, we have to decide which evidence should accompany a result, so that another group can tell which part actually holds. This blog post is about the evidence that should accompany them if they are to become engineering knowledge.

The Scale of the Shift

The workshops reflect a broader rise in AI-mediated systems research. A recent cross-stack survey of more than 7,800 arXiv papers found that the annual AI-for-systems publication volume grew roughly 23× from 2017 to 2025, and even faster in hardware and chip design (GenAI for Systems). These categories reach beyond architecture, and not every paper in them runs an adaptive design loop. But where AI adapts to workloads, simulator feedback, or selection criteria, the final artifact can obscure how the result emerged. As this body of work grows, leaving that process implicit makes it harder to compare results or carry a finding from one group to the next.

Figure 1: Annual AI-for-systems publication volume grew about 23× from 2017 to 2025 (a). The hardware and chip-design categories grew roughly 43× and 60×, respectively, compared with 21× for software (b).

Recent SIGARCH posts show the field working this out in public from different angles. Karu Sankaralingam asks whether architecture has reached an AlphaZero moment, with evaluation, not idea generation, as the real bottleneck. Dimitrios Skarlatos argues that agentic co-design is already reshaping the architect’s role and the hardware-software contract. Jeff Dean and David Patterson project a 10,000× future built on compounding gains, one of them AI automating hardware design itself. Together, their arguments point toward a common question about what should count as evidence when AI helps produce a design. Answering it requires being precise about what changes when AI moves from a bounded tool to an actor in the design process.

What Changes With Agentic Design

AI for architecture means using learned or agentic systems to help shape architectural designs and the evidence used to evaluate them, rather than building hardware optimized to run AI workloads. This direction, framed in recent work on the foundations of AI agents for modern computer system design, overlaps with software generation and electronic design automation (EDA), but it is distinct from both. An open-ended architecture agent can influence mechanisms, abstractions, workloads, simulator configurations, and interfaces whose effects propagate through many downstream programs and tools. That reach is what makes both its designs and its decisions worth scrutinizing.

Machine learning has entered architecture before. Perceptron branch predictors, reinforcement-learning memory controllers, learned prefetchers such as Pythia, surrogate models that approximate expensive simulations, and autotuners that automatically search configuration choices all used statistical learning to sharpen a mechanism or search a design space. In much of that work, ML was part of the artifact or a bounded optimizer, while the workloads, evaluator, and rules governing the search were set outside the model. The agentic shift is not a clean break from autotuning. It expands the scope and authority of the adaptive process. When a system can propose or edit mechanisms, call tools, choose workloads, adapt to simulator feedback, and influence which candidate survives, ML is no longer only inside the design. It starts to shape the claim we make about the design.

This concern predates AI. Human researchers explore design spaces, tune systems, and discard candidates, too, and research has always run on authors disclosing what others need to judge the work, backed by a degree of trust. What changes with an agent is how we scale. An adaptive system can make and revise these choices at machine speed across mechanism code, simulator configurations, workloads, tool calls, and selection criteria, often in response to the same evaluator that later supports the claim. The issue is not that a choice made by an AI system is inherently less trustworthy. It is that a large, tool-mediated search collapses into a final mechanism and a score, and the path that produced them disappears unless someone deliberately records it. The end goal is not to eliminate trust, but to keep that part of the methodology visible enough for others to assess the claim.

Gupta and colleagues’ ArchAgent makes this concrete by designing and implementing cache-replacement policies, not just their parameters. Starting from Mockingjay, a prior state-of-the-art policy, ArchAgent generated Policy31 for the single-core SPEC CPU 2006 suite, with a usage-intensity mechanism that its authors could inspect and test feature by feature. It also generated Policy12, which appeared to beat Mockingjay through what the paper calls a simulator escape, a higher score won by exploiting the simulator rather than the architecture. In ChampSim, unsupported bypassing of last-level cache writes was protected only by an assertion that optimized builds removed, so Policy12 looked faster because the bypassed writes vanished rather than being handled correctly.

The same agentic process produced both a genuine mechanism and a broken measurement, and the reported scores alone would not tell a reviewer which was which. The authors, to their full credit, caught the escape through manual inspection and reported it, exactly the kind of evidence future studies should preserve. A fuller record would not have found the bug automatically, but preserving the build configuration, the rejected policy, and the check that disqualified it would let others see why Policy12 failed and reuse that check in the next study.

Figure 2: (Left) Artifact-only review sees a generated design and a reported number. (Right) Design loop-aware review keeps the artifact in view while adding the declared bounds, the evidence and failures, and a record of who could accept or reject the candidate.

ArchAgent also helps show where Karu Sankaralingam’s AlphaZero comparison holds and where architecture departs from it. AlphaZero discovered powerful Go strategies through self-play, but the board and the rules stayed fixed. Only the strategy could change. Depending on its permissions, an architecture agent can influence the strategy, the board, the rules, and the score used to judge it. Benchmarks can become data the agent adapts to, simulators can become environments it acts on through tool calls, metrics can become optimization targets, and interfaces define what actions it can take. That is why the claim must carry a record of the “design loop,” not just the artifact that emerged from it.

What Evidence to Preserve

Architecture papers already describe mechanisms, baselines, workloads, simulators, and evaluation procedures, so this is not a call for longer methods sections. What they rarely preserve is how the search reached the reported design. A compact record would make a few things visible:

• Bounds: what the agent could see and change, and what stayed fixed
• Feedback: how much simulator feedback it drew on, and how it steered the search
• Evidence and failures: what supported the reported result, and which candidates were rejected and why
• The decision: who could reject a candidate, who made the final call, and what would overturn the result

None of it is exotic. It is the part of the process that an adaptive search tends to erase.

Machine learning has already faced a version of this gap. A released model or dataset often lacked sufficient context to assess its intended use, evaluation, or provenance. In response, model cards and datasheets for datasets provided the field with compact records that accompany the artifact, short enough to read yet specific enough to state what the work does and does not support. Architecture needs the same kind of record, extended from a finished artifact to the search that produced it.

The reporting burden should scale with how much authority the agent had. If AI only helped implement a mechanism specified by a human, ordinary artifact disclosure is probably enough. If it chose workloads, edited the design, adapted to simulator feedback, or determined which candidate was reported, some account of that process should accompany the result. A simple test is whether AI materially shaped the mechanism, workload, evaluator, stopping rule, rejection rule, or reported result. That record need not become a universal checklist, expose the model’s private reasoning, or promise an exact replay of a randomized search. Its format should emerge through use and revision rather than being settled in advance. What matters is how much of an AI-shaped process must stay visible for another group to see why a result survived and whether it holds under different assumptions.

Existing practice offers only partial precedents. Declaring the setup before a search resembles preregistration, keeping the evidence trail resembles artifact evaluation, and preserving failed alternatives resembles ablation studies, which test the effect of changing one part of a design, as well as negative result reporting. The individual practices are not new. The change is that a single adaptive system can operate continuously across the design, workload, evaluator, and stopping rule, which are usually documented separately.

A record like this is a good-faith disclosure, not proof. An author can omit an inconveniently rejected candidate, and a reviewer cannot rerun an adaptive search to catch the omission, especially when the agent relies on a proprietary model that shifts over time and never repeats a run exactly. The record cannot stand on its own. What keeps it honest is disclosure scaled to the agent’s authority, read by reviewers rather than filed as a badge, and confirmed against evidence the search did not produce.

A result selected through adaptive evaluation should face at least one confirmation check outside the search, using held-out workloads, a second simulator, or a targeted test of the claimed mechanism. If the same agent tunes against the simulator that scores it, selects its evaluation workloads, and stops once the metric looks good, the evaluator has become part of the optimization loop, the architecture equivalent of training on the test set, or evaluation leakage. The check must also use measurements appropriate to the claim, and as a recent SIGARCH post on full-system timing simulation argues, simulation speed and fidelity are already in tension before an agent begins optimizing. Agent feedback makes the measurement window and metric part of the search surface, so authors should explain why the confirmation is credible and what evidence would overturn the result.

The same shift that put agents into the design loop is now putting them into the review loop. In systems research, agents already drive the design loop end-to-end, and elsewhere they draft and review their own papers, with a language model serving as the judge. An automated judge can share the blind spots of the system it reviews, so it does not replace the independent check. But it does raise the value of a record built to be machine-readable as well as human-readable, one that the next agent can use to rebuild the setup, rerun the disqualifying check, and test the claim rather than take a summary on faith.

A Shared Layer for the Loop

A record inside a single paper is a good start. It becomes a shared convention when authors use common fields and present supporting evidence in a form others can inspect. A reviewer can then challenge the record, a student can learn why the reported design survived, and another group can revisit a rejected candidate under the same conditions. Some variation in this scaffolding is healthy, but shared infrastructure gives groups a common base without requiring them to pursue the same research questions.

The computer architecture community has previously built shared responses to analogous coordination problems. SPEC provided common workloads, while simulators such as SimpleScalar and gem5 provided researchers with reusable experimental platforms. MLPerf and the long line of prediction and prefetching championships showed that we can agree on workloads, rules, and scoreboards. These shared objects did not settle every question, but they gave the field durable things to run, dispute, teach, and improve. Benchmarks do not capture the path through an adaptive search, but they show how common boundaries make comparisons meaningful. Agentic design now needs a similar layer for search state, allowed actions, failures, and independent checks.

When Amir Yazdanbakhsh and I first articulated the Architecture 2.0 vision in a SIGARCH blog post in 2023, it was conceived as a data-centric AI gymnasium, a shared ecosystem of data, benchmarks, and tools for ML-assisted architecture research. Three years later, many of those building blocks have emerged, including knowledge benchmarks such as QuArch, assembled with the help of more than 140 contributors across 40 institutions, capability evaluations such as ArchEval, and design-space infrastructure such as ArchGym. Building them has taught us something more important. The remaining challenge is not simply another benchmark, evaluation, or piece of infrastructure. A benchmark tests what a model knows, an evaluation tests what an agent can do, and infrastructure runs the search. Some of these systems log a run in detail, but that record stays inside the tool. What no published result yet carries with it is a portable account of how a study bounded its search, rejected candidates, and chose what to report, and that is the part we cannot supply by building one more tool.

Making It Routine

Making this kind of evidence part of everyday research practice will take deliberate community effort. Machine-learning communities have shown one path. Benchmarks and competitions provide participants with shared tasks and rules, while model cards and datasheets establish shared reporting expectations. NeurIPS requires submissions to include a paper checklist addressing reproducibility, transparency, limitations, and experimental details, a step it introduced to help authors document the completeness and limits of their work. ICML has likewise published paper guidelines, based on the NeurIPS checklist, that ask authors to document claims, limitations, code, data, and experimental details. A complementary perspective appears in the 2022 FAccT paper by Smith and colleagues on REAL ML, which argues that responsible machine learning depends not only on models and metrics, but also on documenting the broader research process.

Architecture conferences and workshops could experiment with a few concrete practices. Artifact-evaluation tracks can request versioned configurations, failures that affected the result, and at least one confirmation check that was not used to select the reported result. Competitions can specify workloads, allowed actions, limits on evaluator queries, stopping rules, and held-out tests to ensure scores remain comparable. We do not need to harden these practices into permanent rules at once, and venues can learn what helps reviewers and discard what does not. For proprietary work, the detailed record may remain internal, but a public claim still requires sufficient disclosure for outside groups to assess it. The simplest shared form for that disclosure is a single page. Authors could include or link to a one-page design-loop card summarizing the process in a consistent format.

Figure 3: One possible one-page record makes the bounds, actions, feedback, evidence, failures, and final decision visible.

These conventions also need a public home alongside tools, benchmarks, failure cases, and examples. The Architecture 2.0 hub is one possible starting point. Its value will depend on whether multiple groups use, challenge, revise, and help govern its contents.

No single convention will make AI for architecture an engineering discipline. The value of a shared record is that it lets a result move beyond the group that produced it so someone who was not there can check it, build on it, or challenge it when the evidence does not hold. The full rooms at ISCA were the momentum. Making the evidence travel with the design is what turns momentum into a discipline.

About the Author

Vijay Janapa Reddi is the Gordon McKay Professor of Electrical and Computer Engineering at Harvard University and a visiting professor at ETH Zurich. His work spans computer architecture, machine learning systems, and autonomous agents. He is Vice President and a board member of MLCommons and the author of the open-source Machine Learning Systems book.

Disclaimer: These posts are written by individual contributors to share their thoughts on the Computer Architecture Today blog for the benefit of the community. Any views or opinions represented in this blog are personal, belong solely to the blog author and do not represent those of ACM SIGARCH or its parent organization, ACM.

The conference

The 53rd International Symposium on Computer Architecture (ISCA) was held at the Raleigh Convention Center in Raleigh, North Carolina, from June 27 to July 1, 2026. Raleigh sits at one corner of the Research Triangle, anchored by North Carolina State University, the University of North Carolina at Chapel Hill, and Duke University. General Chairs Huiyang Zhou and James Tuck, both of NC State, led the organizing effort.

The most notable structural change this year was that ISCA offered remote attendance, making it a hybrid conference. The organizers provided deeply discounted remote registration to broaden access for students and researchers who could not travel, broadcast the main and keynote sessions on Zoom, and made recordings available to registrants for offline viewing. This was ISCA’s first hybrid offering and an experiment intended to lay groundwork for remote attendance at future architecture conferences.

Workshops and tutorials

Preceding the main symposium, ISCA 2026 opened with two full days of workshops and tutorials on Saturday, June 27 and Sunday, June 28, organized by Workshops and Tutorials Co-Chairs Lisa Wu Wills (Duke) and Brandon Reagen (NYU). The program totaled 15 workshops and 16 tutorials, spanning the full breadth of the field, from open-source infrastructure and DRAM to quantum computing, encrypted AI, and agentic design.

Saturday’s tutorials leaned on open-source and simulation infrastructure, such as ASTRA-sim, the Ramulator and DRAM Bender memory tools, and FAVA on formal hardware verification, while the workshops included gem5, SAFE AI on encrypted AI, and Architecture 2.0 on agentic AI for computing-systems design, which marked the launch of the book Architecture 2.0: Agentic Design Loops for Computing System Synthesis. Sunday leaned into mentoring, open-source hardware, and quantum: the uArch Mentoring Workshop and YArch’26 for students; XiangShan and the SODA Synthesizer on the open-source side; and FHE & Cheddar and Janus 4.0 for quantum, alongside the 6th DRAMSec, a carbon-accounting tutorial, and the Championship in Branch Prediction.

Panel on the Impact of AI on Higher Education & Computer Architecture @uArch 2026

The main program

This was the largest ISCA program ever. Program Co-Chair Carole-Jean Wu (FAIR, Meta) and Kevin Skadron (University of Virginia) reported 850 regular-track submissions, a 49% increase over the previous year, of which 161 were accepted, for an 18.9% acceptance rate (down from 23% the year before). To accommodate the volume, ISCA ran a fourth parallel track for the first time. 

The reviewing operation scaled to match. It involved 22 Area Chairs, 211 full PC members, and 192 lightweight PC members, by far the largest committee in the conference’s history. Reviewing ran in two rounds, with most papers reaching six reviews. Discussion followed the “Identify the Champion” model; 301 papers reached a clear online consensus, while the remaining 59 were resolved in a series of real-time Zoom PC meetings held over five days. In the end, 116 papers were accepted outright and another 45 were conditionally accepted with shepherding, all of which were eventually accepted.

Keynotes

ISCA 2026 featured three keynotes.

Debbie Marr (CEO and Co-Founder of AheadComputing) opened with “Computing at the Crossroads: Architecture, Economics, and the Next Era.” She reflected on the trajectories that shaped the field: Moore’s Law, Dennard scaling, increasing abstraction, and the long expansion of general-purpose computing. Many of those assumptions, she observed, are now being questioned simultaneously. She tied the technical inflection point to shifting economics, ecosystem dynamics, and leadership transitions, and suggested that the architecture community’s choices today will define the next era of computing.

The second keynote piloted a new “dialogue” format on quantum computing, pairing Fred Chong (University of Chicago; Chief Scientist for Quantum Software at Infleqtion) and Jay Gambetta (IBM Fellow and Director of Research) for “Architecting Hybrid Quantum-Classical Computing for Scale and Fault Tolerance.” Their shared theme: with fault-tolerant machines on the horizon and near-term machines increasingly integrated with classical HPC, computing will be heterogeneous and accelerator-based, and architects are needed to bridge theory and physical technology across applications, software, error correction, workflow management, and machine organization.

Babak Falsafi (EPFL) closed the keynote lineup with “Beyond the AI Energy Wall: Optimal Server Design and Operation”. He described how AI is pushing cloud infrastructure toward an energy wall, with compute demand growing faster than power, cooling, and datacenter capacity can be sustainably provisioned, and suggested that clearing it requires full-stack optimization rather than simply scaling accelerators or building larger facilities. He questioned the long-standing assumption that single-thread performance should dominate server design and operation.

Keynote by Debbie Marr (Computing at the Crossroads)

Awards

A number of the community’s honors were presented during the conference.

The ACM/IEEE-CS Eckert-Mauchly Award went to Srinivas Devadas (MIT) for pioneering contributions to secure architectures with broad industrial and academic impact. The ACM SIGARCH Maurice Wilkes Award was presented to Tushar Krishna (Georgia Tech) for outstanding contributions to architectures and modeling tools for large-scale AI systems. The TCCA Young Architect Award went to Akshitha Sriraman (Carnegie Mellon University) for contributions to the design and management of efficient and sustainable cloud datacenters. The ACM SIGARCH/IEEE CS TCCA Outstanding Dissertation Award went to Olivia Hsu (Stanford University), with an honorable mention to Jovan Stojkovic (University of Illinois Urbana-Champaign). The SIGARCH Alan D. Berenbaum Distinguished Service Award was presented to Sarita Adve (University of Illinois Urbana-Champaign) for sustained and transformative contributions to ACM SIGARCH, the broader architecture community, and via CARES, the ACM SIG ecosystem. The ISCA Influential Paper Award recognized “Adaptive Insertion Policies for High Performance Caching” (ISCA 2007) by Moinuddin K. Qureshi, Aamer Jaleel, Yale N. Patt, Simon C. Steely, and Joel Emer, for its commercial impact and for reinvigorating research on cache management with an elegant set-dueling framework that can be broadly applied to cache-policy selection.

Two ISCA Best Paper Awards were selected from a field of five nominations: “Cerberus: Cross-Layer ECC Co-Design for Robust and Efficient Memory Protection” — Junhwan Kim, Seunghyun Kim, Yesin Ryu, Saeid Gorgin, and Jungrae Kim. “Patterns Behind Chaos: Forecasting Data Movement for Efficient Large-Scale MoE LLM Inference” — Zhongkai Yu, Yue Guan, Zihao Yu, Chenyang Zhou, Zhengding Hu, Shuyi Pei, Yangwook Kang, Yufei Ding, and Po-An Tsai.

Two ISCA Distinguished Artifact Awards were also recognized: “Transpiler-Architecture Co-Design to Curb Clifford Costs in Fault-Tolerant Quantum Computing” — Meng Wang, Chenxu Liu, Samuel Stein, Yufei Ding, Poulami Das, Prashant Nair, and Ang Li. “Towards Practical Interrupt Side-Channel Attacks on macOS for Apple Silicon” — Xin Zhang, Chang Liu, Jiajun Zou, Yi Yang, Qingni Shen, Zhi Zhang, and Trevor E. Carlson.

ISCA Influential Paper Award recipients

Industry track and artifact evaluation

The Industry Track, chaired by Brad Beckmann (AMD), accepted 11 papers out of 27, reviewed by a 32-member committee drawn entirely from industry across a diverse set of startups and established companies. The accepted set ranged from silicon to software. Two additional papers were recommended for an IEEE Micro Special Issue on Commercial Products.

Artifact Evaluation, in its fourth year at ISCA, received 49 submissions. 42 papers earned all three badges (Available, Functional, and Reproduced), 3 earned Available and Functional, and 4 earned Available. The co-chairs Hyeran Jeon (UC Merced), Linghao Song (Yale), and Mark Zhao (University of Colorado Boulder) flagged a growing challenge: the increasing heterogeneity of hardware and software platforms, which reviewers do not always have access to.

A snapshot of the Industry Track session

Excursion

ISCA’s excursion was an evening dinner and social at Raleigh’s historic City Market. Built in 1914 and known for its cobblestone streets and early-twentieth-century lamplight, the district hosted a relaxed, open-air affair, with food stations of North Carolina–inspired dishes, beer, and wine spread across the historic Market Hall, The Grove, and the outdoor spaces between them. With attendees spilling across the market, it made for an excellent networking opportunity and a welcome chance to unwind and catch up with people after the intensity of the technical program.

Excursion venue: City Market

About the author: Bingyao Li is an Assistant Professor in the Computer Science and Engineering Department at the University of California, Riverside. Her research focuses on designing architecture and system features for next-generation GPU platforms and building high-performance LLM infrastructure and systems.

Disclaimer: These posts are written by individual contributors to share their thoughts on the Computer Architecture Today blog for the benefit of the community. Any views or opinions represented in this blog are personal, belong solely to the blog author and do not represent those of ACM SIGARCH or its parent organization, ACM.

Accurate timing simulation remains one of the most important tools in computer architecture, but modern systems have made cycle-level simulation increasingly impractical. Today’s platforms combine many-core CPUs, deep memory hierarchies, accelerators, complex I/O, and large software stacks, making detailed simulation extremely slow—often requiring months to simulate seconds of execution. This “timing simulation wall” has pushed researchers toward approximations such as application-only simulation, fixed instruction windows, or instruction windows representing only the workload. While these reduce runtime, they often sacrifice rigorous end-to-end measurement of real microarchitectural behavior.

This blog argues for a return to rigorous full-system timing simulation—not by simulating everything in detail at all times, but by measuring the right execution intervals, using the right performance metrics, and applying statistically sound methods to make accurate simulation practical again.

Why Full-System Simulation?

Full-system simulation emulates an entire computer system: CPU, memory, devices, operating system, and applications. Unlike user-level simulation, it captures interactions across the full software and hardware stack. Full-system simulation matters because many critical behaviors emerge from OS activity, interrupts, I/O, memory management, synchronization, and device interactions—not from application code alone. Ignoring these layers can misrepresent real system bottlenecks and performance.

Full-system simulation dates back to the 1990s with systems like SimOS, later influencing platforms such as Simics (now Intel Simics Simulator), M5 (integrated into gem5) and QEMU (used in MARSS and QFlex).

Today, full-system simulation is becoming essential again for four reasons:

1. Modern workloads are service-oriented and multi-tenant, relying on microservices, RPCs, storage stacks, and OS-mediated interactions.
2. Many server and mobile workloads spend significant time in the OS, making kernel behavior central to performance analysis.
3. Heterogeneous systems increasingly combine CPUs with GPUs, accelerators, and smart NICs, with the CPU and OS orchestrating coordination, memory, and synchronization.
4. Agentic AI workloads depend heavily on tool invocation, scheduling, APIs, databases, and system integration, making CPU and OS behavior critical to end-to-end performance.

As a result, full-system simulation is no longer just a legacy methodology—it is increasingly necessary because the entire system stack has become the target of architectural innovation.

The Timing Simulation Wall

Simulators span a broad spectrum of abstraction, functionality, and performance. At the fastest end are execution-driven full-system simulators that use JIT translation to dynamically map target ISA instructions into the host ISA at runtime. Since early systems such as SimOS, these simulators have typically operated within roughly an order of magnitude of native hardware speed.

Modern ISA emulators such as QEMU can additionally generate detailed execution traces for functional simulation, enabling analysis of cache and TLB miss rates, branch predictor behavior, and prefetcher accuracy. This tracing introduces another order-of-magnitude slowdown relative to native execution.

Timing simulators go further by modeling cycle-level interactions among microarchitectural components in the CPU, accelerator, memory and I/O devices resulting in substantially lower simulation throughput. The table below compares simulation speeds for a single ARM Neoverse N1 target core with its cache hierarchy running server workloads on an AMD Zen 3 host.  The first row presents QEMU’s raw ISA emulation speed. The second row shows the slowdown due to instrumentation for user-level functional simulation. The third row demonstrates the impact on speed when functionally simulating the microarchitectural components, including the cache hierarchy and TLBs, front-end tables, and data prefetcher, for all user-level instructions. The fourth row shows the impact of functional simulation of all instructions, including the OS. Finally, the fifth row shows the timing simulation speed.

Modern workloads are not steady streams of similar instructions. Their performance fluctuates over time due to network activity, resource contention, background OS activity, synchronization effects, software hiccups, DVFS throttling, UI and graphics activity, and other asynchronous events. Alameldeen et al. presented a statistically rigorous methodology to determine the minimum measurement window needed to capture workload performance variability within a specified error bound and confidence level. 

Unlike conventional database workloads (e.g., TPC benchmarks) which have prescribed measurement windows, typical benchmarks and workloads used in research do not. Applying Alameldeen’s methodology, we find that capturing performance variability for a single ARM Neoverse N1 core and its cache hierarchy requires five to 120 seconds of target execution time across server workloads from CloudSuite, DCPerf, and DeathStarBench. Simulating even a few seconds of a single core with today’s fastest cycle-accurate simulator, gem5, at 250 KIPS requires months of simulation time.

What Should We Measure?

The second question is which performance metric to use. Timing simulators count cycles, so architects often report IPC, or instructions per cycle. IPC is reasonable for single-core workloads when most executed instructions correspond to program progress.

For multicore workloads, however, IPC can be misleading. Threads may spin, poll, block, wait on locks, synchronize, or execute OS code that does not advance useful work. A system can therefore sustain high IPC while making little forward progress; in effect, total IPC can reward busy waiting. 

This is why user-level IPC, or U-IPC, is often a better proxy. U-IPC counts user-level instructions over time, assuming that user instructions per request remain roughly stable and that most spinning occurs in the OS. Under that assumption, U-IPC tracks useful throughput more directly than total IPC.

But U-IPC must be validated for each workload. If spinning occurs in user space, as in systems with user-level network stacks, raw U-IPC still counts non-productive work and must be corrected to exclude spinning. The broader requirement is therefore metric validation: a rigorous simulation methodology must show that the chosen metric—IPC, U-IPC, throughput, or latency—actually captures forward progress for the workload under study.

How Should We Measure?

Due to the timing simulation wall, researchers often use abbreviated measurements. The most common technique is to measure a single unit of 100 million to one billion instructions. Unfortunately, depending on where in the execution the fixed measurement is taken from, this technique may lead to inconclusive results or worse, incorrect conclusions. 

Instead, designers often use sampling to capture variability in performance estimates. Phase-based sampling, such as SimPoint, is a popular technique that uses clustering of basic-block vectors (BBVs) to select representative application “phases.” Such sampling properly captures the representing repetitive instruction streams that account for most of the execution. 

While simple and practical, phase-based sampling may ignore OS effects, interrupts and I/O interactions, communication among cores, and software hiccups. Moreover, Wunderlich argues in his thesis that phase-based sampling: (1) misses the microarchitectural footprint of less common instruction streams and their impact on performance, and (2) forgoes any error bounds with confidence in estimates.

A rigorous sampling technique is statistical sampling, such as SMARTS, taking a large sample (e.g., hundreds) of small (e.g., 200k cycles), uniformly distanced measurement units that is representative of execution, not phases in the workload. This technique enables bounding the error in estimates and delivers quantifiable confidence. It also opens an entire plethora of statistical sampling tools to trade off confidence in estimates for measurement in speed and quantify sample divergence to detect bias in estimates.

The figure below compares error magnitude in performance estimates among three abbreviated measurement techniques from full-timing simulation runs of tens of target seconds on a two-core socket with 2.0 GHz ARM Neoverse N1 cores running single-tier, multi-tier and consolidated server workloads (CloudSuite, DCPerf and DeathStarBench). The figure compares the error against the full-timing baselines for: (1) single units of one billion instructions per core starting from three equally distanced positions in the minimum measurement window (i.e., beginning, 1/3 and 2/3 into the population), (2) units of 100 target microseconds (i.e., 200k cycles for a 2.0 GHz clock) including basic-block vectors (BBV) derived from K-means clustering, and (3) a uniform sample (of hundreds) of 100 target microseconds drawn with an error bound of 5% with 95% confidence with statistical sampling. 

Both one-billion instruction units and BBV result in high error estimates with the former not being representative of execution and the latter representing only frequently executed instructions. In contrast, statistical sampling results in a desired error bound with confidence because it represents not just frequently executed instructions but also instructions that have a high impact on performance due to their microarchitectural footprint.

A SOTA Sampling Framework

The figure below presents a state-of-the-art sampling framework using QFlex 3.0 (derived from SimFlex) for full-system timing simulation of ARM ISA. For each workload, the software stack together with the OS is first loaded and warmed on a real platform, then tested to identify the minimum window—such as five to 120 target machine’s seconds—called a “population”, using Alameldeen et al.’s technique. The workload is then loaded again, this time with QEMU and run through a functional simulator running on average at 6 MIPS for the entire duration of population. 

The functional simulator simulates all microarchitectural components with long-term state (e.g., cache hierarchy and TLBs, branch tables, data prefetcher) and periodically dumps checkpoints with architectural and microarchitectural state into a checkpoint library. Because the functional simulator is not cycle-accurate, it requires an approximation for time. The most common approximation is IPC=1 or IPC derived from neighboring units.

The checkpoints in the library are then run using a timing simulator for 100 us independently and embarrassingly parallel. Each checkpoint is first run for a bounded window of time (e.g., 200 us) to make sure microarchitectural components with short-term state (e.g., buffers in the pipeline, cache hierarchy and NoC) are warm, followed by a measurement. The timing results for the sample are then aggregated to determine whether the sample (i.e., number of checkpoints) is large enough to bound the error for a desired level of confidence (e.g., 5% with 95% confidence). If not, the sampling framework creates a new checkpoint library with a shorter interval between checkpoints.

Challenges and Open Problems 

Even with accurate measurement techniques, there are fundamental challenges with sampling (for both phase-based and statistical sampling).

1. Accurate state generation. Timing-induced activity during functional simulation and its impact on the microarchitectural footprint may result in a significant bias because time is approximated. This challenge is more pronounced with variable performance among target threads in multi-tier and consolidated workloads where the speed bias may significantly impact the resulting shared microarchitectural footprint.
2. The functional simulation wall. Sampling minimizes the required measurement using timing simulators but shifts the bottleneck to the functional simulator (which at 6 MIPS is only 24x faster than a 250 KIPs timing simulator). Parallelizing functional simulation may be a promising approach to enable scalability with multicore hosts. Parallel simulation is fundamentally limited by the granularity at which target threads communicate.
3. Support for checkpointing. Generating and restoring an entire checkpoint for every measurement is impractical in both storage capacity and runtime overhead. Practical sampling therefore requires incremental checkpoint storage and restoration.
4. Sampling non-average metrics. Statistical sampling works well for average-like metrics such as IPC or U-IPC, but it is harder to apply to extreme or rare-event metrics such as maximum temperature, worst-case power, or rare latency spikes.
5. Capturing service-level metrics. Metrics such as request latency or p99.9 latency are much coarser-grained than sampling units needed for IPC or U-IPC. Capturing service-level metrics and tail latency may require an order of magnitude larger population and sampling units which poses a challenge for both functional and timing simulation.
6. Multi-node full-system simulation. Many modern workloads are distributed across multiple machines. Single-node simulation is often insufficient for datacenter-scale behavior, but despite progress, rigorous  multi-node full-system timing simulation remains an open challenge.
7. Interoperability across simulators. A practical ecosystem should allow one tool to generate a checkpoint library and another to perform timing simulation. This interoperability requires an interface definition language allowing interoperable architectural and microarchitectural state among simulators.

About the Authors

Shanqing Lin is a final-year PhD student at the School of Computer and Communication Sciences at EPFL and the principal developer of QFlex v3.0.

Mohammad Alian is an Assistant Professor in the Electrical and Computer Engineering Department at Cornell University.

Babak Falsafi is a Professor in the School of Computer and Communication Sciences at EPFL (epfl.ch) and the founding President of Swiss Datacenter Efficiency Association (sdea.ch).

Disclaimer: These posts are written by individual contributors to share their thoughts on the Computer Architecture Today blog for the benefit of the community. Any views or opinions represented in this blog are personal, belong solely to the blog author and do not represent those of ACM SIGARCH or its parent organization, ACM.

What does speculative execution in a processor — and the predictor that drives it, such as a branch predictor — have to do with AI agents? They may seem very different, yet, at a high level of abstraction there are similarities.

Both speculate: A processor predicts which way a branch will go and begins executing instructions along the predicted path before the branch has resolved. An AI agent infers a user’s intent, reads/writes files, executes programs, makes network calls etc., before it knows whether its interpretation of the user’s intent is right.

Because prediction can fail, both systems require roll back mechanisms. In a processor, once a misprediction is detected, the wrong path work disappears (from the programmer’s point of view). When the agent is told it is wrong, or figures that out itself, it may revise its plan and redo the work after rolling back to a good checkpoint.

Both systems litter and leave residues: While a processor can recover from a misprediction without any programmer visible effects, under the covers, microarchitecturally, wrong path execution perturbs on chip structures like caches. The Spectre attack (2018) showed that this residue can be observed through covert channels. AI agents have a similar problem. When an agent, or its human user, notices a mistake and corrects it, the failed attempt can leave at least two types of residues: a) residue that is easily observable like bad outputs, stale files or processes, or b) harder to know/track/undo residue like timing and volume of network requests, model context summaries shipped to third party servers to name a few.

Also both systems can be tricked and steered through adversarial inputs: In Spectre, the attacker influences the on chip predictor state by executing a pattern, then supplies an adversarial input that causes the victim to transiently execute along the trained path that it should not take architecturally. While that transient execution is later squashed the microarchitectural residue of the execution can still be measured. Malicious prompts can play a similar role in AI agents: they can steer the system toward actions that are later corrected or denied, but in the process may leave litter data that attackers can use.

Given these similarities, we can ask two questions.

1) Can AI agents completely eliminate easily observable “architectural” residues on mispredictions?

2) What are the dangers/risks of hidden “microarchitectural” residue left behind by AI agents?

Regarding architectural residue, processors can hide speculative wrong path work cleanly because the ISA defines what counts as visible committed state. Currently there is no equivalent for AI agents: the absence of an interface that can precisely define operations, state, life time of state, and triggers for misprediction recovery, makes these systems hard to reason about and a fertile ground for leakage.

While observable residue is a serious problem it is also a solvable problem to some degree: if one is satisfied with imprecise, best effort work, one simple thing to do is to just prompt the agent to clean up after itself. A really smart agent, in theory, should be able to use mechanisms like transactions, two phase commit, distributed undo protocols, disposable containers and VMs, sandboxes, access controls, versioning and information flow tracking to minimize overt residues. However, if we wanted to do better than prompting we probably will need an ISA-like layer.

The second, and harder, question is about what happens to hidden/microarchitectural residues. In general, clean up of this type of residue is hard because it is often left in places no one thinks to inspect, or in places users cannot practically inspect because the those parts are proprietary or distributed across organizational boundaries. Also with AI agents, the problem is broader in scope than in a processor because it spans a larger number of tech layers from model context to hardware, local and remote. Further, an agent’s speculation window may last seconds or minutes, compared with nanoseconds in a processor. That longer window creates more opportunity for residue to diffuse. It is highly unlikely that we can simply prompt the agent to clean up hidden/microarchitectural residue because, by definition, there isn’t an architectural interface to observe or control microarchitectural state/work.

How likely are we to solve agentic littering? Who needs this problem solved? And, who should solve this problem?

In addition to technical aspects, economics and incentives often determine whether solutions are adopted. Here too we can look at processor misprediction recovery and compare them to AI agents.

Overt architectural and hidden microarchitectural residues have different economics and incentives at play.

Overt residues are easier to price. If an agent leaves behind a directory full of junk, consumes too many resources, or corrupts a file, that failure is visible to users. Users will complain, and because there are complaints, product teams can justify spending resources to fix them.

Hidden residues are harder. These residues may not produce an obvious effect like a crash. They may also require complex conditions to manifest. That makes it harder to attribute with accuracy and consequently easier to dismiss. It also makes it harder for users to demand fixes, because users often cannot see the thing they are supposed to complain about.

Spectre, an issue due to adversarial steering and microarchitectural residue, was disclosed roughly eight years ago, and the broader class of this leakage has still not been completely fixed. This is not because principled technical solutions do not exist. It is because these solutions increase design complexity, impact performance, change the hardware and software interface in ways that is not easy to adopt, or require coordination across vendors and different layers of the computing stack all of which add recurring or non-recurring costs. Also, each layer can plausibly say that the residue cleanup should be handled by someone else. Vendors can also say that there are have not seen large scale attacks and that they do not have to protect against these attacks given the risk profile.

The same pattern may emerge for AI agents and handling hidden/microarchitectural residues.

Each AI agent boundary is also an economic boundary. Each layer can plausibly say that the residue is someone else’s problem. The model provider can say the deployment should isolate side effects. The deployment/orchestrator can say the runtime should enforce cleanup. The runtime can say the operating system should provide better isolation. The hardware vendor can say software should avoid sensitive colocation. The user experiences the combined risk of all these but usually has the least ability to inspect or repair it!

The real answer is that every party involved in agentic execution should fix its own leaks and share responsibility for security and privacy. But each party also has reason to argue that the cost is too high, especially when the economic benefits are difficult to measure and the harms are difficult to attribute.

So the likely outcome here is not hard to guess. Hidden microarchitectural residue handling is treated as an afterthought, and agents end up reflecting the incentives that shaped it, viz., agents get more capable, overt residue cleanup improves through ad hoc clean up attempts, and create a very long tail of hard to detect, microarchitectural residues that expands the attack surface.

The best chance for security is while these systems are being designed and deployed. AI-agent platforms designed now should at least treat residue management as first-class design requirement. That, however, means finding ways to incentivize designers to care about hidden, microarchitectural residue before users are harmed. If we treat microrchitectural residue management as an optional, “nice-to-have”, “less-important-than-overt” security feature, we will spend the next decade patching a massive, distributed attack surface.

About the Author: Simha Sethumadhavan is a Professor in the CS department at Columbia University. He would like to thank  Profs. Roxana Geambasu. Martha Kim and Tanvir Ahmed Khan for thought provoking comments and feedback.

Disclaimer: These posts are written by individual contributors to share their thoughts on the Computer Architecture Today blog for the benefit of the community. Any views or opinions represented in this blog are personal, belong solely to the blog author and do not represent those of ACM SIGARCH or its parent organization, ACM.