Written Tue 6 Oct 2026, evening, on the Intel. It answers three asks you dictated after the camera arrived: 1. Can a lower voltage let a card run without its heatsink? I read your "with heat sink" as "without". 2. Can the card come off the motherboard on a longer cable, so the CPU's tower cooler stops blocking the camera? 3. How should the P3 be mounted apart from the phone?
It builds on the lab's 30 Sep no-heatsink study (et-soc1-prototyping). That study already modelled the lowest voltages the firmware accepts. No card was touched for this page.
Part of the heatsink and thermal-camera group in the lab's observability hub (§1). The other pages in that group are listed at the end.
The short answers
- Lower voltage alone won't let a card run bare. At the firmware's lowest voltages and a 100 MHz clock, the 30 Sep model puts aifoundry2's idle power at 19 W at a 60 °C die, down from 26 W, or about a quarter less. The rest comes from parts no minion voltage reaches. Going below the firmware's floor would save about 1–1.5 W more, at real risk.
- What could make it work is cooling that leaves the lid in view. Most of a bare package's heat goes into the board, not out through the lid. So the candidates are: - a heatsink pressed to the back of the board, behind the chip; - colder intake air; - a strong air jet on the lid.
One safe test tells you whether this is worth trying, with the heatsink still on (§2). 3. Yes, the card can come off the motherboard, but try the free move first. aifoundry2's board has a second Gen4 slot, PCIEX8, about 6 cm farther from the CPU cooler. The card keeps its 16 GT/s x8 link there, with no BIOS change.
If that isn't enough, use a Gen5-rated riser, or better a cable kit whose card-end adapter takes its own 12 V cable. That cable is also where a power meter and a hard cut-off go (§3). The "Xeon" is the CPU's tower cooler; the hosts run Core i5 and i7 chips. 4. Mount the P3 on its own arm and run a USB cable to it; for long runs, plug it into a small Linux box instead of the phone. The P3 comes with a 48–50 cm extension cable for exactly this. Past about a metre, make the long stretch a USB-A extension (§4).
1. Lower voltage: what the firmware allows, and why it isn't enough
What can be set. The lowest voltages the firmware accepts are:
| Rail | Lowest accepted |
|---|---|
| Minion | 400 mV |
| SRAM | 660 mV |
| NoC | 400 mV |
| DDR | 700 mV |
| Maxion | 600 mV |
- The lowest clock is 100 MHz.
- The vendor's own lowest operating point is its "safe state": 300 MHz with the minion at 400 mV and the SRAM at 660 mV.
- aifoundry1's card 0 already idles there, at about 18 W.
What it does to idle power. These are aifoundry2's idle figures from the 30 Sep model, in watts at the board. The columns are die temperatures.
| Setting | 25 °C | 40 °C | 60 °C |
|---|---|---|---|
| 600 MHz, voltages as booted (518/704 mV) | 17.4 | 20.0 | 25.8 |
| 100 MHz, voltages as booted | 15.5 | 18.2 | 24.0 |
| 100 MHz, lowest voltages (398/660 mV) | 14.2 | 16.2 | 20.4 |
| 100 MHz, lowest voltages, NoC slowed to 200 MHz at 400 mV | 13.7 | 15.5 | 19.2 |
Why it stops there. At idle, 59% of the card's power (18.7 of 31.8 W at 73 °C on aifoundry2) is the NoC rail plus things on no metered rail: - the service processor; - the PCIe link; - the memory shires and their DRAM, where power-down and self-refresh are switched off in the firmware; - the Maxion rail.
No minion clock or voltage touches any of that.
Below the floor. The regulators can go as low as 250 mV, and card 1's old 0.18.0 firmware checks nothing. But it buys little and risks a lot: - From 400 mV, every further 50 mV cuts the minion rail's leakage by only about a quarter. That's about 1 W at the 300 MHz point, so 1–1.5 W in total. - The SRAM rail already sits only 40–45 mV above its 660 mV floor, and no retention voltage is documented anywhere. - On the 0.20.0 firmware, a rejected voltage write can loop until the watchdog resets the card. - The 0.18.0 firmware writes every NoC setting to flash.
Esperanto's own papers point the same way. The Hot Chips 33 and IEEE Micro papers put the best efficiency near 0.3 V, at 8.5 W a chip. But those were design projections for a running workload, not idle figures. The lab's real silicon needs 517 mV at 600 MHz.
2. Cooling that keeps the lid in view
A card settles only if its cooling beats the way its leakage grows with temperature. Here is that cooling budget, as the largest thermal resistance (°C per watt of the chip's own power) at which an idle aifoundry2 still settles at or below 85 °C:
| Setting | Air at 25 °C | Air at 15 °C | Air at 10 °C |
|---|---|---|---|
| 600 MHz, as booted | 1.8–2.0 | 2.2–2.4 | 2.4–2.6 |
| 100 MHz, lowest voltages, slow NoC | 2.5–2.7 | 3.0–3.3 | 3.2–3.6 |
The 25 °C column is the lab model's (1.83–2.02 and 2.57–2.93). The colder columns come from the same law of leakage against temperature, calibrated to that column and to today's measured 27.4 W at 64 °C.
What each set-up gives (°C/W):
| Set-up | Thermal resistance | Source |
|---|---|---|
| Heatsink on, as today | about 1.9 (1.5 per watt at the board): close to the 600 MHz limit, which is why aifoundry2 ran away before today's fan fix | measured fit |
| Bare lid, still air | 4.4–6.5 | lab model |
| Bare lid, a 1–2.5 m/s fan | 2.3–3.8 | lab model |
| Bare lid, fan, and a heatsink on the board's back | about 1.3–3.7 | my estimate; the spread is the package's unknown junction-to-board resistance |
| Bare lid, a strong air jet (20–60 m/s), and a back heatsink | about 1.0–2.7 | my estimate |
Read the two tables together:
- A fan alone fails at 25 °C at either clock, which is the 30 Sep answer.
- Colder air is worth as much as the lowest voltages. With an ordinary fan, the lab's physical model puts a bare lid at about 2.9 °C/W. At 100 MHz with the lowest voltages, that settles at about 50–60 °C in 10–15 °C air, and runs away in 25 °C air. Two facts from the lab's own runaway analysis show why air temperature matters so much:
- With its heatsink on, aifoundry2 rests at 62 °C in 22.8 °C air.
- In 28 °C air it has no resting point at all.
- A heatsink on the board's back is the cheapest big step.
- In still air about 90% of a bare package's heat leaves through its balls into the board, and only about 10% through the lid.
- Karlsruhe's IR-thermography rig (Amrouch and Henkel, ISLPED 2015) did the same thing at the die level: it cooled the chip from the rear so the camera could see the front.
- The catch is the package's junction-to-board resistance. Nobody publishes it for this package; the lab model assumes 1–3 °C/W.
The safe test, heatsink still on. Clamp a finned heatsink with a fan to the back of the board, behind the package: for example a 60 mm aluminium sink with a 60 mm fan. Use a soft, electrically insulating gap pad, 1–2 mm thick, so it sits over the decoupling capacitors. Then read the idle die temperature against today's 64 °C at 27.4 W.
| Idle drop | What it means for bare running |
|---|---|
| About 15 °C or more | The back can carry enough heat for 100 MHz at the lowest voltages |
| About 20 °C or more | Even 600 MHz looks possible |
| Under about 8 °C | The board's back can't carry it; stay with the heatsink |
These thresholds are rough. They assume today's board-to-air path is 4–6 °C/W. Either way the test leaves the card cooler than it is now, and this is the card that ran away to 138 °C.
Update, Fri 9 Oct 2026 (week 41): these thresholds were set for the cooling of 6 Oct, about 1.9 °C/W. Since the new fans of 9 Oct (0.68 °C/W, idle 51 °C), the same test drops the idle by only 3–6 °C: about 6 °C means the package passes heat into the board well, 3 °C or less means poorly. Worked out in cooling from the back, with the camera on top.
Two more things a bare run would need, both already in the 30 Sep plan or close to it: - A power cut that doesn't depend on the host. A riser makes one possible (§3). - Booting straight into the low point. Today a card idles at 600 MHz while the host boots. That is why the model's bare "look" at 100 MHz lasts a median of 0–6 s, and why half of power-ons give none. A firmware that boots into the 300 MHz safe state, as card 0's already does, removes that window. That's a question for Roman.
What a bare lid would show. The 30 Sep imaging model found the following: - A single frame of the lid shows nearly one temperature. - Lock-in brings out a single shire and coarse patterns within about 10 s at 100 MHz. - With the heatsink on, the chip's own 34 sensors see the same contrast between neighbouring shires as the bare lid would. Today's firmware reports only their mean; per-sensor readings are request CF14 for Roman.
3. Moving the card off the motherboard
First, the free move: the PCIEX8 slot. This is from the Gigabyte Z590 AORUS MASTER manual; aifoundry2's board was read from its DMI table.
| Slot | Position | Link |
|---|---|---|
| PCIEX16 (the card's slot now) | 1, next to the CPU | Gen4 from the CPU |
| PCIEX8 | 4, about 61 mm farther away | Gen4 from the CPU |
| PCIEX4 | 6 | Gen3 x4 from the chipset: not usable |
- The two CPU slots split the CPU's 16 lanes, x8 and x8. The card only uses x8, so in PCIEX8 it still trains at 16 GT/s x8.
- The 61 mm comes from scaling the manual's drawing: exactly three slot pitches.
- Check on site:
- that the heatsink and its 120 mm fan fit below PCIEX8;
- whether 6 cm more gap opens enough of the card's back to the camera.
- aifoundry1's and aifoundry3's boards haven't been read; your photos show AORUS boards.
If that isn't enough: a riser cable or a cable kit.
| Route | Example | Length | Price | Note |
|---|---|---|---|---|
| Ribbon riser, Gen5-rated | LINKUP AVA5 | 15–150 cm | $64 (15 cm) to $124 (60 cm) | the most margin at the card's Gen4 |
| Ribbon riser, Gen5-rated | Lian Li PW-PCI-520X | 20 cm | about $65 | |
| Ribbon riser, Gen4 | LINKUP Ultra | 14–38 cm | about $30 | |
| Cable kit, Gen5 parts | C-Payne MCIO: passive x16 host adapter, x8/x16 device adapter, 8i cable | per cable | about €140 | the device adapter takes its own 12 V cable |
| Cable kit, Gen4 | C-Payne SlimSAS: x8 host adapter with redriver, device adapter, 8i cable | 45 or 75 cm | about €120 | same |
- Length matters less than you'd think. Hardware Canucks tested more than 20 Gen4 ribbon risers (review). Passes and failures followed no pattern, and the length "didn't seem to matter"; the motherboard did. So buy Gen5-rated parts for margin, and plan to verify on this board.
- Power is the real difference between the two kinds.
- A ribbon riser carries the card's 12 V on thin ribbon wires. The PCIe slot spec allows 5.5 A (66 W) on 12 V.
- The card is rated 7.3 A (88 W) at its edge, and its heaviest load on record was 87.8 W.
- Idle (about 2.3 A) and the usual 64 W load (about 5.3 A) are within the slot spec; the peaks aren't.
- A cable kit's device adapter feeds the card from its own 12 V cable from the power supply, so nothing heavy runs through a ribbon or the motherboard.
- Its maker warns that the wrong power connector "will destroy the Adapter and your Device", so use the cable it ships with.
- That 12 V cable is where a power meter and a hard cut-off go. The lab's notes already want kilohertz power readings from a riser. The 30 Sep plan already wants a cut that doesn't depend on the host before any bare run.
- ElmorLabs sells a $30 Gen5 slot adapter (ADT-Link R33G). It takes the card's slot 12 V from an external 6+2-pin cable, read by its PMD2X meter ($109). The 3.3 V still comes from the host.
- Cutting the card's 12 V under a running host is a surprise removal, and the host may hang. It's the last line of defence, not the routine stop.
- Two lab rules carry over:
- Move the card only with the host cold. Never retrain a running link.
- Write down the root port's corrected-error count in today's slot before the move, then compare. After it,
lspci -vvshould still readLnkSta: Speed 16GT/s, Width x8. - If the link turns out marginal, the BIOS's CPU PCIe Link Speed (Settings → Miscellaneous) can force Gen3. It applies to every CPU-attached slot, including the CPU's M.2.
- The fan moves with the card. Out of the case's airflow, keep the 120 mm fan on its heatsink at full speed. Today's fix was exactly that, after the idle card ran away.
- One thing to settle before re-routing power. The lab's notes disagree on whether the card's ATX auxiliary 12 V input passes through its current sense:
- the 30 Sep page says it joins after;
- the power-telemetry notes say a jumper puts either source through it.
If it joins after, the card's own power readings would stop covering the whole card. Check the card manual first.
Update, Fri 9 Oct 2026: the card's V3 drawing (in AI Foundry's et-man repo) shows the aux input joining after the current sensor and its protection switch, so power fed through it is invisible to the card's own readings. Confirm with a meter before relying on it. - Mounting the card: a vertical GPU holder or a printed test-bench stand; the job is strain relief on the cable, not weight. - Others have done it. - Tanel Põder ran an RTX 4090 on a PCIe 4.0 extender for five months with no AER errors. - Igor's Lab puts a shunt riser in the slot beside an Optris IR camera, the same pairing of power meter and camera.
4. Mounting the P3 apart from the phone
In the box: - A USB-C extension cable, 48–50 cm, male to female. Gough Lui says it works "under all circumstances" to place the camera independently of the phone. - A USB-C-to-USB-A adapter. Leave it in the box: an EEVblog reviewer traced intermittent disconnects to its shallow plug. - A Lightning adapter and a case. - No tripod thread and no clip.
Running a longer cable. The P3 is a USB 2.0 device drawing about 0.3 W, so length isn't a real problem at its data rate. The USB-C rule against extension cables is about 10–20 Gb/s signals and charging power, neither of which the P3 uses. For 1–3 m, chain these in order:
- the phone
- the Anker 332 hub, which you need anyway: the phone can't charge through its port while the camera is plugged in
- a 2–3 m USB 2.0 A-to-A extension (passive is within spec to 5 m; an active repeater beyond that)
- a good USB-C-female-to-USB-A-male adapter, not the one in the box
- the 48–50 cm extension from the box
- the camera
The box's short cable at the camera end takes the strain, so you can re-aim the camera without disturbing the run.
The mount: - The camera. Use a printed cover with a GoPro-style back, MakerWorld 2283135, plus a $5 GoPro-to-1/4"-20 adapter. That goes on the SmallRig 2065 arm and 2058 clamp from the 3 Oct list. I couldn't open the MakerWorld page itself, so check the fit before printing. - MakerWorld 2903257 is a handheld grip, but its oversized focus knob is worth copying. - Thermal Master also sells a "6D inspection stand" bundle for bench work. - Where to clamp it. Clamp to a shelf post or a separate stand, not to the PC's open frame. At 0.3–1 mm per pixel, fan vibration smears the image. Since then: the 9 Oct thermal camera hub prefers the frame's rear wall, so the camera moves with the card. Friday thermal measurements (14 Oct) puts a bracket in the open slot 2, with the lens 52–65 mm from the card's back. - Distance. At 10 cm the P3 sees 7.3 × 5.4 cm at 0.28 mm per pixel, so the 45 mm package spans about 160 pixels. At 20 cm it sees 14.6 × 10.8 cm. At 40 cm it sees roughly the whole card.
For long runs: - Keep it out of the exhaust. A microbolometer drifts with its own temperature. - Warm it up. Give it 20 minutes and throw that data away. - Use high gain (−20 to 150 °C), and gaffer-tape the focus ring. - Mark the shutter gaps. The camera re-zeros itself about every 90 s, dropping 1–2 frames; log those so lock-in skips them. - Tape what you measure. Shiny metal reads the room, so read black Scotch 88 vinyl tape (emissivity 0.96) or Kapton. Corrected Wed 14 Oct: 1-mil Kapton is partly transparent to the camera, about 0.62 over metal, so prefer the vinyl; see calibrating the P3.
Better than the phone for long runs: a Linux box.
- Josh Dillon's driver (p3-ir-camera; Python 3.10+, pyusb, one udev rule) gives raw 16-bit temperatures at 25 fps. It's about 5 MB/s raw, so decimate what you log.
- A fork, thermal-camera-viewer (Apache-2.0, updated 21 Sep 2026), installs as a .deb on Ubuntu, Debian or Raspberry Pi OS. It makes the camera appear as an ordinary webcam, /dev/video10, so any streaming tool can serve it over the tailnet.
- Two candidate hosts:
- A Raspberry Pi 5 next to the rack, which your 27 Sep plan already named.
- The lab host itself. It then sits on the same clock as the workload, so lock-in timing comes free. It needs Roman's OK for a package and a udev rule on a shared box.
- On a Mac, the driver reportedly runs with just brew install libusb (issue #3), which is handy for trying it at home first.
- Expect crashes. The driver has had USBError crashes (issue #6, closed with a workaround), so a multi-hour logger needs a supervisor that reconnects.
- The phone stays the walk-around and demo tool.
Who has to say yes
- Roman or Nate:
- moving a card or changing slots;
- a riser;
- anything clamped to the card;
- a package or udev rule on a lab host;
- a firmware that boots into the safe state.
- Never on card 1: any voltage write. Its 0.18.0 firmware writes NoC settings to flash.
- Before any bare run: the 30 Sep plan's stops and a power cut tested end to end.
The heatsink and thermal-camera pages
Grouped as "Heat, the heatsink and the thermal camera" in the observability hub's §1, since Wed 7 Oct 2026 (week 41).
The lab's pages (public):
- Thermal camera hub, 9 Oct. Every thermal page since 22 Sep, newest first.
- Friday thermal measurements, 14 Oct. The space behind aifoundry2's card, measured from the 9 Oct photos and drawn to scale.
- Feasibility of running the ET-SoC-1 without its heatsink, 30 Sep. The question this page follows up; its byline and related list now link back here.
- Pointing a thermal camera at the ET-SoC-1, 26 Sep. Your camera plan and lockin.py.
- The effect of overheating, 28 Sep.
- Where the work sits: placement and the thermal trip, 27–28 Sep.
- aifoundry2's card is back, 6 Oct. The fan fix after an idle card ran away with its heatsink on.
Yours (private): - A thermal camera that streams live through your Pixel, 3 Oct. - P3 on chips: what to watch and read, 6 Oct.
Sources
- The lab's 30 Sep study: page; data and firmware notes (
lowclock_calc.out§B–F,nohs_calc.out§B–D,LOWCLOCK-FIRMWARE.md§3–5,LOWCLOCK-PLAN.md§9,LOWCLOCK-OUTSIDE.mdO1–O3). - aifoundry2's measured idle after today's fan fix: fan-fix report.
- The lab hosts' hardware: et-soc1-prototyping
14-card-behaviour.mdand the 27 Sep chip-diagram research. - Amrouch and Henkel, Lucid Infrared Thermography of Thermally-Constrained Processors, ISLPED 2015.
- P3: Gough Lui's review, the EEVblog thread, PCWorld,
P3_PROTOCOL.md. - Microbolometer warm-up: arXiv 2302.05198.
- The board: Gigabyte Z590 AORUS MASTER manual rev. 1002, pp. 10, 55 and 58 (mirror).
- Risers and cable kits: Hardware Canucks, C-Payne device adapter, lspci link checks (Intel).
- The slot power spec: Intel.
- Emissivity tape: FLIR.
- My two calculations (
theta_needed.py,rear_sink_test.py) and their printouts: in the lab repo atdocs/reports/data/2026-10-06-imaging-followup/, and here atreports/2026-10/data/2026-10-06-et-soc1-imaging/. - Earlier pages: the 3 Oct camera rig, P3 reading list.
Total life satisfaction: the camera is the week's goal you named at this morning's standup. The fastest path to a first picture carries no new risk to the chip: the card in the PCIEX8 slot or on a riser, the heatsink on, and the camera on its own arm looking at the back of the board.