Difference between revisions of "OHSNAP"

From Noisebridge
Jump to: navigation, search
m (link to cor1a)
Line 20: Line 20:
  
 
<h2>Status</h2>
 
<h2>Status</h2>
 +
<h3>August 10, 2021</h3>
 +
[details to come]
 +
 +
<h3>August 3, 2021</h3>
 +
Meeting notes with [https://www.sifive.com SiFive] / [https://openfive.com OpenFive]:
 +
* OpenFive has the capabilities to package both custom and standard cores to customer specs and can easily put the UC74-MC into a QFP package.
 +
* Costs for ''any'' ASIC work are divided between IP and manufacturing.
 +
** Manufacturing cost scales inversely with process node.
 +
*** 28nm process is roughly $1M for masks and another $1M for tapeout + packaging + validation. This is for a typical/minimum order quantity of 10,000.
 +
*** For comparison, 5nm processes are on the order of $20M+.
 +
** The maximum process node is determined by design requirements (particularly max frequency, die size, power consumption, thermal dissipation). For OHSNAP Router our estimated lower frequency is ~700 MHz. This most likely means a maximum of 28nm process.
 +
** Re-spins are often necessary for most clients due to silicon bugs. For 28nm, this adds a typical $150-200k for a 3 layer respin.
 +
** Typical time between submitting final RTL designs and start of tapeout is 1 year. This includes 1 week to generate masks, 10-12 weeks to process the silicon (for 28nm), postprocessing time (may not be necessary for QFP packages), die assembly at the package assembly site, and testing. Add another 4-5 months until we can receive engineering samples. Total time for the entire process from start to ramp is 1.5-2 years.
 +
** OpenFive uses two foundries: TSMC and GlobalFoundries. Our requirement for domestic manufacturing eliminates TSMC and 6 out of the 9 GlobalFoundries sites, but the remaining 3 are viable candidates. According to [https://en.wikipedia.org/wiki/GlobalFoundries#Fabrication_plants this article], two of those can hit 28nm and both are in New York.
 +
** IP cost depends on what core we use and what options we select. The stock [https://www.sifive.com/cores/u74-mc UC74-MC core] that we were considering is about ~$1M. Additional features drive the price up. Switching from -MC (multicore) to [https://www.sifive.com/cores/u74 single core] only saves a modest amount of money.
 +
* They offered a lower-end alternative to the U7-series: The [https://www.sifive.com/cores/u54 U5 series]. It is also Linux capable but has a shorter pipeline and so isn’t as performant. Typical IP is $500k.
 +
* Tamper evidence / manufacturing control:
 +
** Three main arenas to consider: Silicon manufacturing, packaging + assembly, and testing.
 +
** OpenFive transports the products themselves.
 +
** They would be open to us installing an observer to monitor production (for a fee).
 +
** They would be open to providing us the results of Layout-Versus-Schematic (LVS) tests.
 +
* Development cycle
 +
** 1. Pre-silicon: Emulation
 +
*** QEMU
 +
*** RTL
 +
*** FPGA Bitstream
 +
*** System C
 +
** 2. Dev board
 +
*** For U5/U7 series chips: [https://www.sifive.com/boards/hifive-unmatched HiFive Unmatched]
 +
*** Do bulk of development while waiting for engineering samples
 +
** 3. Engineering samples
 +
*** Finish main development
 +
*** Debug debug debug
 +
*** Possibly re-spin Rev B silicon
 +
* They briefly mentioned [https://www.sifive.com/blog/sifive-shield-an-open-scalable-platform-architecture SiFive Shield and WorldGuard], two architectural technologies for enhanced silicon security. While these don’t solve our problem of trusted manufacturing, they are definitely useful layers to add to our secure ecosystem.
 +
* They seemed ready to engage with us but the cost is the big question. They were impressed by our mission and our passion and wanted to know more about the device, its use cases, and our business model. We told them that they are our Plan A and we really want to work with them but their cost estimates were an order of magnitude greater than we anticipated so we have to go back to the drawing board.
 +
 +
<h3>July 27, 2021</h3>
 +
Preparing for meeting with SiFive to discuss custom packaging of the UC74-MC RISC-V core.
 +
 
<h3>July 9, 2021</h3>
 
<h3>July 9, 2021</h3>
 
Explored RISC-V option. Many advantages:
 
Explored RISC-V option. Many advantages:
Line 141: Line 181:
 
<h3>March 20, 2021</h3>
 
<h3>March 20, 2021</h3>
 
<p>In order to make forward progress on the hardware, we will choose OpenBSD as the initial operating system and an ARM-based architecture for the CPU. Users will be able to use their own ARM-compatible OS if they choose, including Linux and Plan9 (if software support is added).</p>
 
<p>In order to make forward progress on the hardware, we will choose OpenBSD as the initial operating system and an ARM-based architecture for the CPU. Users will be able to use their own ARM-compatible OS if they choose, including Linux and Plan9 (if software support is added).</p>
<p>We will first create a Software-Intent Proof of Concept (SIPoC), which will be an OpenBSD-based router running on a commercially available SBC. We may then want, as a Proof of Concept (PoC), to port the software stack to the Common Networks nodes ([[COR1A]] at Noisebridge and begin deploying them across sites.</p>
+
<p>We will first create a Software-Intent Proof of Concept (SIPoC), which will be an OpenBSD-based router running on a commercially available SBC. We may then want, as a Proof of Concept (PoC), to port the software stack to the [[COR1A|Common Networks nodes at Noisebridge]] and begin deploying them across sites.</p>
 
<p>The first version of our custom hardware should be amendable to DIY manufacturing. This means no BGA parts. This severely restricts the list of CPUs/SoCs to older and lower-performance chips, limiting our capability to 100MBps initially. We'll expand this list as we discover more:</p>
 
<p>The first version of our custom hardware should be amendable to DIY manufacturing. This means no BGA parts. This severely restricts the list of CPUs/SoCs to older and lower-performance chips, limiting our capability to 100MBps initially. We'll expand this list as we discover more:</p>
 
<h4>List of non-BGA ARM SoCs</h4>
 
<h4>List of non-BGA ARM SoCs</h4>

Revision as of 23:22, 10 August 2021

Open Hardware for Secure Networks And Privacy (OHSNAP)

This is the project page for OHSNAP, an open-source platform for building secure networks with a known hardware root of trust.

Motivation

Virtually all commercially-available networking equipment is proprietary and closed-source and cannot be independently verified to be free of malware. There have been documented cases of attackers – sometimes entire nation-states – physically modifying networking equipment and networkable devices in order to exfiltrate data and/or command & control otherwise-trusted systems. This leaves the average individual with little choice but to hope that their home network consists of and is secured by devices which do not phone home or contain other backdoors. Such a situation breaks the guarantee that the user's data and devices remain their sovereign property and instead places control into the hands of manufacturers and governments.

The goal of this project is to produce completely open designs and implementations for critical network infrastructure with verifiable roots of trust. By making the hardware design, manufacturing process, and firmware and software stacks fully verifiable, it allows users to inspect the entire end-to-end flow of their data and to directly control some or all of the fabrication of the device in order to establish positive provenance.

Device Summary

OHSNAP is an exploratory platform for secure hardware. The first two OHSNAP devices will be:

  • OHSNAP Router: An embedded computer running an open source firmware and OS. It will expose at least two gigabit Ethernet ports.
  • OHSNAP Server: A small, low-power, single-board computer with a single 10/100 Ethernet port.

Design Goals

  • No closed-source firmware or software allowed anywhere in the stack
  • Implementation must be independently reproducible by third parties
  • Factory-made PCBs must be physically produced in the USA
  • Components should be as supplier-diversified as possible

Status

August 10, 2021

[details to come]

August 3, 2021

Meeting notes with SiFive / OpenFive:

  • OpenFive has the capabilities to package both custom and standard cores to customer specs and can easily put the UC74-MC into a QFP package.
  • Costs for any ASIC work are divided between IP and manufacturing.
    • Manufacturing cost scales inversely with process node.
      • 28nm process is roughly $1M for masks and another $1M for tapeout + packaging + validation. This is for a typical/minimum order quantity of 10,000.
      • For comparison, 5nm processes are on the order of $20M+.
    • The maximum process node is determined by design requirements (particularly max frequency, die size, power consumption, thermal dissipation). For OHSNAP Router our estimated lower frequency is ~700 MHz. This most likely means a maximum of 28nm process.
    • Re-spins are often necessary for most clients due to silicon bugs. For 28nm, this adds a typical $150-200k for a 3 layer respin.
    • Typical time between submitting final RTL designs and start of tapeout is 1 year. This includes 1 week to generate masks, 10-12 weeks to process the silicon (for 28nm), postprocessing time (may not be necessary for QFP packages), die assembly at the package assembly site, and testing. Add another 4-5 months until we can receive engineering samples. Total time for the entire process from start to ramp is 1.5-2 years.
    • OpenFive uses two foundries: TSMC and GlobalFoundries. Our requirement for domestic manufacturing eliminates TSMC and 6 out of the 9 GlobalFoundries sites, but the remaining 3 are viable candidates. According to this article, two of those can hit 28nm and both are in New York.
    • IP cost depends on what core we use and what options we select. The stock UC74-MC core that we were considering is about ~$1M. Additional features drive the price up. Switching from -MC (multicore) to single core only saves a modest amount of money.
  • They offered a lower-end alternative to the U7-series: The U5 series. It is also Linux capable but has a shorter pipeline and so isn’t as performant. Typical IP is $500k.
  • Tamper evidence / manufacturing control:
    • Three main arenas to consider: Silicon manufacturing, packaging + assembly, and testing.
    • OpenFive transports the products themselves.
    • They would be open to us installing an observer to monitor production (for a fee).
    • They would be open to providing us the results of Layout-Versus-Schematic (LVS) tests.
  • Development cycle
    • 1. Pre-silicon: Emulation
      • QEMU
      • RTL
      • FPGA Bitstream
      • System C
    • 2. Dev board
      • For U5/U7 series chips: HiFive Unmatched
      • Do bulk of development while waiting for engineering samples
    • 3. Engineering samples
      • Finish main development
      • Debug debug debug
      • Possibly re-spin Rev B silicon
  • They briefly mentioned SiFive Shield and WorldGuard, two architectural technologies for enhanced silicon security. While these don’t solve our problem of trusted manufacturing, they are definitely useful layers to add to our secure ecosystem.
  • They seemed ready to engage with us but the cost is the big question. They were impressed by our mission and our passion and wanted to know more about the device, its use cases, and our business model. We told them that they are our Plan A and we really want to work with them but their cost estimates were an order of magnitude greater than we anticipated so we have to go back to the drawing board.

July 27, 2021

Preparing for meeting with SiFive to discuss custom packaging of the UC74-MC RISC-V core.

July 9, 2021

Explored RISC-V option. Many advantages:

  • We can reduce the attack surface by including only the silicon IP blocks that we need
  • Eliminates the requirement for outsourcing PCB manufacture
  • One chip can serve as the base platform for all OHSNAP projects (router, server, TFC, etc.) as well as being adoptable by third parties
  • Potentially improves best-case root-of-trust guarantee (but this requires a lot of thought and care)
  • Extends the idea of Open Hardware beyond traditional limits!

Main barriers:

  • High cost of entry
  • Need vendor buy-in for enhanced tamper evident manufacturing practices
  • Significant development effort

We will deliberate and vote on this. If in consensus, next step is to engage vendor(s) and start understanding the height of the above barriers.

July 2, 2021

Examination of the OpenBSD source and mailing list archives shows that support for pre-v7 ARM architectures was intentionally removed from the OS. According to this table, the NUC980's ARM926EJ core is ARMv5-based and therefore unsupported.

Adding support for ARMv5 to OpenBSD is out of scope for this project and we're not aware of any plans by the OpenBSD developers to do so either. This means that we can't use the NUC980 with OpenBSD for the OHSNAP Server and leaves us with these choices:

  1. Pause the OHSNAP Server project and continue to focus on OHSNAP Router
  2. Resume searching for OpenBSD-compatible non-BGA chips that are similar to the Nuvoton (unlikely to exist on the market, as we have been searching for a while and chips like this tend to be older designs and therefore unlikely to be supported)
  3. Migrate to a different OS which has ARM926EJ support, such as NetBSD or Linux. Both are considered less secure than OpenBSD, the latter unacceptably so.
  4. With newly added and quickly expanding RISC-V support in OpenBSD, we may want to consider that architecture more seriously. One key advantage of open source silicon is that we can create a custom, high-performance chip and put it in a QFP package with only the pins exposed that we would need. Next steps would be getting ballpark estimates of taping out our own silicon from local vendors.

June 25, 2021

  • Created v0.1 schematic for NUC980-based dev board. Will first be used with bare metal firmware.
  • Ordered samples of NUC980DR61Y and the Nuvoton Chili dev board. The latter will be used to test firmware and Linux and to begin the OpenBSD port.
  • Found article on porting NetBSD to ARM926EJ-S, the same core as in the Nuvoton chip.

June 18, 2021

We've settled on a roadmap:

  1. Create a fully homebuilt, low-performance 10/100 single Ethernet board that is capable of running OpenBSD ("Option A" from March 27, 2021). This approach will:
    1. Provide us with hands-on experience in designing, building, debugging our own hardware.
    2. Give us a testbed for experimenting with secure manufacturing practices.
    3. Require us to learn how to port OpenBSD to a new chip and board.
    4. Provide a ready-to-use reference design for other, related secure manufacturing projects at Noisebridge and beyond.
    5. Produce a physical deliverable that will create buy-in and grow our ecosystem.
  2. In parallel, we will continue to identify and engage potential vendors for high-performance designs which require securely outsourcing manufacturing.

The first low-performance design will be built around the Nuvoton NUC980DR61Y (see March 20, 2021). This will come in stages:

  1. HWPOC (Hardware Proof Of Concept): A minimal board designed to allow creating bare metal firmware to test basic design and in-house manufacturing on the Voltera PCB printer. We may attempt to boot Linux from this board, as there is kernel support for it and the design will be derived from the Nuvoton Chili dev board.
  2. SWPOC (Software Proof of Concept): Use the Nuvoton Chili board to port OpenBSD to the target chip.
  3. Proto: Boot OpenBSD from the minimal in-house board.
  4. EVT: First in-house board featuring Ethernet and all planned features. Still contains debugging connections.
  5. DVT: Finalized PCB. Begin testing low-qty mass production using pick & place machine.
  6. PVT: Final production units. Goes on to ramp build.

May 8, 2021

  • Continued exploration of threat vectors and possible mitigations.
  • More links added to resource list.
  • Setting up Tinfoil Chat as a realtime collaboration platform and exploration of secure methodologies.
  • Setting up OpenBSD on various SBCs and virtual machines to evaluate its fitness for use.

April 3, 2021

Identified three regimes for threat models, some concrete examples, and some hypothetical mitigations:

Threat Regime Example Scenario Possible Mitigations
During assembly Manufacturer tampering
  • PCB x-ray inspection
  • Translucent PCB substrate for visual inspection
  • Encase PCB in glitter epoxy
  • Tamper-evident paint
  • Apply heat-sensitive paint to PCB (detects soldering)
  • Vacuum-sealed enclosure with onetime-use pressure sensor
  • Serial numbers in ROM on ICs
  • Use common enough chip to reduce chance of silicon tampering
  • Pogo board testing
During transit to end-user Vendor/reseller tampering
Mail interdiction
Embed device in another device
At customer's install site Evil user tampering Live system intrusion detection (examples??)
Routine pogo board testing
Passive monitoring, e.g.,
  • audio/video monitoring of keypresses
  • RF leakage analysis
  • power analysis
???

March 27, 2021

Still looking for a non-BGA ARM processor that contains two Ethernet PHYs and has 0.5mm or greater lead pitch (for Voltera V-1 PCB production. We are now considering three options for design:

  • (A) Non-BGA ARM processor on a homemade mainboard [simplest but limited to non-gigabit speeds; may require SW porting]
  • (B) BGA ARM processor on a fabbed carrier board attached to a homemade mainboard [more complicated but higher performance and easier to source; may avoid SW porting]
    • E.g., NXP i.MX6SoloX
    • Unpopulated carrier board will need to be examined, probably via x-ray, after production
    • Carrier board reflow is still done by us
    • Still need to look for more ways to increase provenance
  • (C) Use an existing open source BGA-based project (e.g., Kosagi Novena) and modify it for trusted manufacturing [complexity likely between (A) and (B)]
    • Will still likely need a BGA carrier board from (B) but may reduce R&D costs of the project

Infrastructure

We’ll be setting up an IRC server in parallel to testing local deployment of the Element secure chat system.

March 20, 2021

In order to make forward progress on the hardware, we will choose OpenBSD as the initial operating system and an ARM-based architecture for the CPU. Users will be able to use their own ARM-compatible OS if they choose, including Linux and Plan9 (if software support is added).

We will first create a Software-Intent Proof of Concept (SIPoC), which will be an OpenBSD-based router running on a commercially available SBC. We may then want, as a Proof of Concept (PoC), to port the software stack to the Common Networks nodes at Noisebridge and begin deploying them across sites.

The first version of our custom hardware should be amendable to DIY manufacturing. This means no BGA parts. This severely restricts the list of CPUs/SoCs to older and lower-performance chips, limiting our capability to 100MBps initially. We'll expand this list as we discover more:

List of non-BGA ARM SoCs

March 13, 2021

  • Looking for secure communications platform for project collaboration. Element / Matrix look promising.
  • Possibly partner with CircuitLaunch for local hardware builds?

March 6, 2021

Initial meeting. Discussed range of HW/SW design choices.

Tentative Project Stages

  • SIPoC: OpenBSD router on commercial SBC
  • PoC: SW stack on Common Networks
  • Proto 1 build: Low-speed (10/100 Mbps) DIY version
  • Full build: 1 Gbps

Possible Design Choices for Future Versions

  • CPU
    • ARM/ARM64 SoC
    • RISC-V SoC
    • FPGA
    • Specifically no Intel/compatible architectures due to poor security record
  • OS / Application Code
    • OpenBSD
    • Qubes
    • Alpine Linux
    • Plan9
    • Custom FPGA code
  • Trusted manufacturers

Open Questions

  • How to offer root-of-trust guarantees to non-DIY customers

Meetings

We are currently meeting every Tuesday at 16:30 PT (GMT-8) on the Noisebridge Jitsi video platform.

Links to Resources