Overview

Porthos is optimized to sacrifice single threaded performance for efficiency, so that a design is achieved that is realizable in terms of silicon area and clock frequency. We refer to such an architecture as Massive Multithreading, or MMT. A maximum of one instruction can be executed from each of the threads depending on instruction dependencies and availability of resources. Porthos can achieve a sustained execution rate of 40 instructions per cycle (5 instructions per cycle per tribe). The tribe pipeline structure that allows this is shown below.

Each tribe consists of three decoupled pipelines. The first is the instruction fetch pipeline in which, in a classic SMT fashion, two threads are selected for fetch in each cycle and each can fetch four instructions from a shared data cache. Thus the peak fetch rate is eight instructions per cycle per tribe (64 instructions per cycle across the entire chip). The second pipeline represents an SMP type of structure in which each thread is mostly independent. Each thread block contains a small ALU and a single ported register file. The third pipeline is the shared execute pipeline, in which complex ALU instructions and memory instructions are executed. This pipeline is a classic SMT pipeline in which three threads simultaneously execute in each cycle. Even though the peak execution rate is three instructions in the third pipeline, the sustained execution rate is greater since branch instructions and some simple ALU instructions are fully executed in the second pipeline.

The use of a general purpose ISA as well as other techniques achieves a design in which software porting issues are minimized. Thread synchronization is achieved utilzing an innovative mechanism in which threads waiting for short-term synchronization events simply stall and don't consume any global execution resources for busy wait. We use the term "Massive Multithreading", or MMT, to refer to processors with most of the following characteristics:

• A General Purpose ISA (not microengines)
• Hardware support for large numbers of threads (100s rather than 2-8)
• Singled threaded performance has been sacrificed, e.g.
  • no branch prediction
  • in-order issue
  • no speculative fetch
  • no speculative execution
  • single port (or less) to the register file per thread
  • no ALU bypass logic (dependent operations pass operands through register file)
  • a maximum of one operation per thread issued per cycle

The following chart illustrates, that under certain assumptions for power consumption and memory latency, an optimal performance to power ratio can be achieved with single threaded performance well below the maximum achievable using current superscalar microarchitectures.

That is, even though it is possible to build pipelines that can support less than one cycle per instruction through the use of wide issue windows, out of order execution and branch prediction, when there are many threads available, this is wasteful in power. An optimal point is shown here to be on the order of four to five cycles per instruction. In practice, a peak CPI in the range of two to three was chosen as optimal taking into account packet dependencies.

For Further Reading

Melvin, S., Nemirovsky, M., Musoll, E., Huynh, J., Milito, R., Urdaneta, H., and Saraf, K., "A Massively Multithreaded Packet Processor," Workshop on Network Processors - NP2, Held in conjunction with the 9th International Symposium on High-Performance Computer Architecture, February 8-9, 2003. Slides to Presentation

(A version of this paper also appears as Chapter 6 of Network Processor Design: Issues and Practices, Volume 2, Morgan Kaufmann, 2003)

Published U.S. Patent Application US20030069920A1

Various Historical Multithreaded Machines and Research

• 1970: I/O Processor of the CDC 6600 (J. Thornton)
• 1981: Denelcor HEP-1 (B. Smith)
• 1984: Denelcor HEP-3 (aka Vulture) (S. Melvin, B. Smith (unpublished, but acknowledged in: Thistle, M. and Smith, B. A Processor Architecture for Horizon, 1988))
• 1988: Supercompting Research Center Horizon Project (J. Kuehn, B. Smith)
• 1990: Tera Computer System (R. Alverson, D. Callahan, D. Cummings, B. Koblenz, A. Porterfield, B. Smith)
• 1991: Multiple Instruction Stream Processors (Section 7.2.1), pp. 126 - 128 (Ph.D. Dissertation, S. Melvin)
• 1994: Performance Tradeoffs in Multistreamed Superscalar Architectures (Ph.D. Dissertation, M. Serrano)
• 1994: Multistreamed Superscalar Processors (W. Yamamoto, M. Nemirovsky)
• 1995: Simultaneous Multithreading (D. Tullsen, S. Eggers, H. Levy)
• 1999: Compaq Alpha EV-8 (21464) (J. Emer)
• 2000: Clearwater (XStream Logic) CNP810SP
• 2001: Intel Hyperthreading
• 2001: Sun SMT Processor
• 2003: Massive Multithreading (S. Melvin, M. Nemirovsky, E. Musoll, J. Huynh, R. Milito, H. Urdaneta, K. Saraf)

A Brief History of FlowStorm, Inc.

About the Author