OHSNAP: Difference between revisions

From Noisebridge
Jump to navigation Jump to search
No edit summary
No edit summary
(24 intermediate revisions by 2 users not shown)
Line 1: Line 1:
<h1>Open Hardware for Secure Networks And Privacy (OHSNAP)</h1>
<h1>Open Hardware for Secure Networks And Privacy (OHSNAP)</h1>


This is the project page for OHSNAP, an open-source platform for building secure networks with a known root of trust.
This is the project page for OHSNAP, an open-source platform for building secure networks with a known hardware root of trust.


<h2>Motivation</h2>
<h2>Motivation</h2>
<p>Virtually all commercially-available networking equipment is proprietary and closed-source and cannot be independently verified to be free of malware. There have been documented cases of attackers – [https://www.bloomberg.com/news/features/2018-10-04/the-big-hack-how-china-used-a-tiny-chip-to-infiltrate-america-s-top-companies sometimes entire nation-states] – physically modifying networking equipment and networkable devices in order to exfiltrate data and/or command & control otherwise-trusted systems. This leaves the average individual with little choice but to hope that their home network consists of and is secured by devices which do not phone home or contain other backdoors. Such a situation breaks the guarantee that the user's data and devices remain their sovereign property and instead places control into the hands of manufacturers and governments.</p>
<p>Virtually all commercially available networking equipment is proprietary and closed-source and cannot be independently verified to be free of malware. There have been documented cases of attackers – [https://www.bloomberg.com/news/features/2018-10-04/the-big-hack-how-china-used-a-tiny-chip-to-infiltrate-america-s-top-companies sometimes entire nation-states] – physically modifying networking equipment and networkable devices in order to exfiltrate data and/or command & control otherwise-trusted systems. This leaves the average individual with little choice but to hope that their home network consists of and is secured by devices which do not phone home or contain other backdoors. Such a situation breaks the guarantee that the user's data and devices remain their sovereign property and instead places control into the hands of manufacturers and governments.</p>


<p>The goal of this project is to produce a completely open design and implementation for a network router with a verifiable root of trust. By making the hardware design, manufacturing process, and firmware and software stacks fully verifiable, it allows users to inspect the entire end-to-end flow of their data and to directly control some or all of the fabrication of the device in order to establish positive provenance.</p>
<p>The goal of this project is to produce completely open designs and implementations for critical network infrastructure with auditable roots of trust. By making the hardware design, manufacturing process, and firmware and software stacks fully auditable, it allows users to inspect the entire end-to-end flow of their data and to directly control some or all of the fabrication of the device in order to establish positive provenance.</p>


<h2>Device Summary</h2>
<h2>Device Summary</h2>
The OHSNAP router will be a single-board computer running an open source firmware and OS. It will expose at least two Ethernet ports.
<span id="firewall-diagram">[[File:ohsnap_firewall_block_diagram_v0.11.png|right|600px|alt=OHSNAP Firewall platform block diagram]]</span>
<span id="server-diagram">[[File:ohsnap_server_block_diagram_v0.1.png|right|600px|alt=OHSNAP Server platform block diagram]]</span>
OHSNAP is an exploratory platform for secure hardware. The first two OHSNAP devices will be:
* '''OHSNAP Firewall''': An embedded computer running an open source firmware and OS. It will expose at least two gigabit Ethernet ports and is intended to be a trusted checkpoint between two untrusted networks.
* '''OHSNAP Server''': A small, low-power, single-board computer with a single 10/100 Ethernet port. May be used as a server, client, or IoT device.


<h2>Design Goals</h2>
<h2>Design Goals</h2>
* No closed-source firmware or software allowed anywhere in the stack
* No closed-source firmware or software allowed anywhere in the stack
* Implementation must be independently reproducible by third parties
* Implementation must be independently reproducible by third parties
* Factory-made PCBs must be physically produced in the USA
* Factory-made PCBs must be avoided if possible or physically produced in the USA when necessary
* Components should be as supplier-diversified as possible
* Components should be as supplier-diversified as possible
* Every phase of manufacturing must be either tamper-proof or tamper-evident
* Fewer pieces = smaller attack surface
<h2>Meetings</h2>
<p>We are currently meeting every Thursday at 16:30 PT (GMT-8) on the [https://jitsi.noisebridge.io/ohsnap Noisebridge Jitsi video platform].</p>


<h2>Status</h2>
<h2>Status</h2>
<span id="server-timeline">[[File:ohsnap_server_timeline_v0.1.png|right|600px|alt=OHSNAP Server engineering timeline]]</span>
<h3>August 10, 2021</h3>
* Work on Nuvoton dev board PCB layout is in progress; expected checkpoint in September
* Identified current main need: '''We need to grow the team to help solve major root-of-trust questions'''
** Need an easily digestible web presence (like a one-pager with an easy URL) to quickly show prospective collaborators
*** '''We need a web designer to help with this!'''
** Move to a more distributed collaboration platform (Element/Matrix)
** Get more visuals (diagrams, photos) of plan and works-in-progress
*** Added [[#firewall-diagram|block diagram for OHSNAP Firewall]]
** Continue to reach out to potential contributors and allies
*** '''Can YOU help?'''
* Discussed alternatives to SiFive/OpenFive + related projects
** [https://www.skywatertechnology.com Skywater] - low-cost foundry with 180nm and 130nm nodes
*** [https://github.com/google/skywater-pdk Skywater PDK]
** [https://efabless.com Efabless] is offering free (in conjunction with Google) and low-cost custom silicon
** [https://libre-soc.org Libre-SOC] - open source silicon SoC
** [http://parallel.princeton.edu/openpiton/ Princeton OpenPiton] - open source research microprocessor
** [https://www.youtube.com/watch?v=EczW2IWdnOM Tim 'mithro' Ansell] gives an overview of open source silicon tools
<h3>August 3, 2021</h3>
Meeting notes with [https://www.sifive.com SiFive] / [https://openfive.com OpenFive]:
* '''Can they do it?''' Yes: OpenFive has the capabilities to package both custom and standard cores to customer specs and can easily put the UC74-MC into a QFP package.
* '''Costs''' for ''any'' ASIC work are divided between IP and manufacturing.
** Manufacturing cost scales inversely with process node.
*** 28nm process is roughly $1M for masks and another $1M for tapeout + packaging + validation. This is for a typical/minimum order quantity of 10,000.
**** For comparison, 5nm processes are on the order of $20M+.
*** The maximum process node is determined by design requirements (particularly max frequency, die size, power consumption, thermal dissipation). For OHSNAP Firewall our estimated lower frequency is ~700 MHz. This most likely means a maximum of 28nm process.
*** Re-spins are often necessary for most clients due to silicon bugs. For 28nm, this adds a typical $150-200k for a 3 layer respin.
** IP cost depends on what core we use and what options we select. The stock [https://www.sifive.com/cores/u74-mc UC74-MC core] that we were considering is about ~$1M. Additional features drive the price up. Switching from -MC (multicore) to [https://www.sifive.com/cores/u74 single core] only saves a modest amount of money.
*** They offered a lower-end alternative to the U7-series: The [https://www.sifive.com/cores/u54 U5 series]. It is also Linux capable but has a shorter pipeline and so isn’t as performant. Typical IP is $500k.
* '''Time''': Typical time between submitting final RTL designs and start of tapeout is 1 year. This includes 1 week to generate masks, 10-12 weeks to process the silicon (for 28nm), postprocessing time (may not be necessary for QFP packages), die assembly at the package assembly site, and testing. Add another 4-5 months until we can receive engineering samples. Total time for the entire process from start to ramp is 1.5-2 years.
* '''Tamper evidence / manufacturing control''':
** OpenFive uses two foundries: TSMC and GlobalFoundries. Our requirement for domestic manufacturing eliminates TSMC and 6 out of the 9 GlobalFoundries sites, but the remaining 3 are viable candidates. According to [https://en.wikipedia.org/wiki/GlobalFoundries#Fabrication_plants this article], two of those can hit 28nm and both are in New York.
** Three main arenas to consider: Silicon manufacturing, packaging + assembly, and testing.
** OpenFive transports the products themselves.
** They would be open to us installing an observer to monitor production (for a fee).
** They would be open to providing us the results of Layout-Versus-Schematic (LVS) tests.
* '''Development cycle'''
** 1. Pre-silicon: Emulation
*** QEMU
*** RTL
*** FPGA Bitstream
*** System C
** 2. Dev board
*** For U5/U7 series chips: [https://www.sifive.com/boards/hifive-unmatched HiFive Unmatched]
*** Do bulk of development while waiting for engineering samples
** 3. Engineering samples
*** Finish main development
*** Debug debug debug
*** Possibly re-spin Rev B silicon
* They briefly mentioned [https://www.sifive.com/blog/sifive-shield-an-open-scalable-platform-architecture SiFive Shield and WorldGuard], two architectural technologies for enhanced silicon security. While these don’t solve our problem of trusted manufacturing, they are definitely useful layers to add to our secure ecosystem.
* '''Summary:''' They seemed ready to engage with us but the cost is the big question. They were impressed by our mission and our passion and wanted to know more about the device, its use cases, and our business model. We told them that they are our Plan A and we really want to work with them but their cost estimates were an order of magnitude greater than we anticipated so we have to go back to the drawing board.
<h3>July 27, 2021</h3>
Preparing for meeting with SiFive to discuss custom packaging of the UC74-MC RISC-V core.
<h3>July 9, 2021</h3>
Explored RISC-V option. Many advantages:
* We can reduce the attack surface by including only the silicon IP blocks that we need
* Eliminates the requirement for outsourcing PCB manufacture
* One chip can serve as the base platform for all OHSNAP projects (firewall, server, TFC, etc.) as well as being adoptable by third parties
* ''Potentially'' improves best-case root-of-trust guarantee (but this requires a lot of thought and care)
* Extends the idea of Open Hardware beyond traditional limits!
Main barriers:
* High cost of entry
* Need vendor buy-in for enhanced tamper evident manufacturing practices
* Significant development effort
We will deliberate and vote on this. If in consensus, next step is to engage vendor(s) and start understanding the height of the above barriers.
<h3>July 2, 2021</h3>
Examination of the OpenBSD source and mailing list archives shows that support for pre-v7 ARM architectures was [https://marc.info/?l=openbsd-tech&m=145747444923566&w=2 intentionally removed] from the OS. According to [https://en.wikipedia.org/wiki/List_of_ARM_microarchitectures this table], the NUC980's ARM926EJ core is ARMv5-based and therefore unsupported.
Adding support for ARMv5 to OpenBSD is out of scope for this project and we're not aware of any plans by the OpenBSD developers to do so either. This means that we can't use the NUC980 with OpenBSD for the OHSNAP Server and leaves us with these choices:
# Pause the OHSNAP Server project and continue to focus on OHSNAP Firewall
# Resume searching for OpenBSD-compatible non-BGA chips that are similar to the Nuvoton (unlikely to exist on the market, as we have been searching for a while and chips like this tend to be older designs and therefore unlikely to be supported)
# Migrate to a different OS which has ARM926EJ support, such as NetBSD or Linux. Both are considered less secure than OpenBSD, the latter unacceptably so.
# With [http://www.undeadly.org/cgi?action=article;sid=20210423090342 newly added] and [http://www.undeadly.org/cgi?action=article;sid=20210619161607 quickly expanding] RISC-V support in OpenBSD, we may want to consider that architecture more seriously. One key advantage of open source silicon is that we can create a custom, high-performance chip and put it in a QFP package with only the pins exposed that we would need. Next steps would be getting ballpark estimates of taping out our own silicon from [https://www.sifive.com local] [https://openfive.com/custom-silicon/ vendors].
<h3>June 25, 2021</h3>
* Created v0.1 schematic for NUC980-based dev board. Will first be used with bare metal firmware.
* Ordered samples of NUC980DR61Y and the Nuvoton Chili dev board. The latter will be used to test firmware and Linux and to begin the OpenBSD port.
* Found [http://www.netbsd.org/docs/kernel/porting_netbsd_arm_soc.html article] on porting NetBSD to ARM926EJ-S, the same core as in the Nuvoton chip.
<h3>June 18, 2021</h3>
We've settled on a roadmap:
# Create a fully homebuilt, low-performance 10/100 single Ethernet board that is capable of running OpenBSD ("Option '''A'''" from [[#March 27, 2021|March 27, 2021]]). This approach will:
## Provide us with hands-on experience in designing, building, debugging our own hardware.
## Give us a testbed for experimenting with secure manufacturing practices.
## Require us to learn how to port OpenBSD to a new chip and board.
## Provide a ready-to-use reference design for other, related secure manufacturing projects at Noisebridge and beyond.
## Produce a physical deliverable that will create buy-in and grow our ecosystem.
# In parallel, we will continue to identify and engage potential vendors for high-performance designs which require securely outsourcing manufacturing.
The first low-performance design will be built around the [http://www.nuvoton.com.cn/products/microprocessors/arm9-mpus/nuc980-industrial-control-iot-series/nuc980dr61yc/ Nuvoton NUC980DR61Y] (see [[#March 20, 2021|March 20, 2021]]). This will come in stages:
# HWPOC (Hardware Proof Of Concept): A minimal board designed to allow creating bare metal firmware to test basic design and in-house manufacturing on the [https://www.voltera.io Voltera PCB printer]. We may attempt to boot Linux from this board, as there is kernel support for it and the design will be derived from the [https://www.nuvoton.com/products/iot-solution/iot-platform/numaker-rtu-nuc980 Nuvoton Chili] dev board.
# SWPOC (Software Proof of Concept): Use the Nuvoton Chili board to port OpenBSD to the target chip.
# Proto: Boot OpenBSD from the minimal in-house board.
# EVT: First in-house board featuring Ethernet and all planned features. Still contains debugging connections.
# DVT: Finalized PCB. Begin testing low-qty mass production using pick & place machine.
# PVT: Final production units. Goes on to ramp build.
<h3>May 8, 2021</h3>
* Continued exploration of threat vectors and possible mitigations.
* More links added to [[#Links to Resources|resource list]].
* Setting up [https://github.com/maqp/tfc Tinfoil Chat] as a realtime collaboration platform and exploration of secure methodologies.
* Setting up OpenBSD on various SBCs and virtual machines to evaluate its fitness for use.
<h3>April 3, 2021</h3>
<p>Identified three regimes for threat models, some concrete examples, and some hypothetical mitigations:</p>
<table style="border: 1px solid black; " cellspacing="3" bgcolor="#000000">
    <tr bgcolor="#ffffff">
        <th style="padding: 1em;">Threat Regime</th>
        <th style="padding: 1em;">Example Scenario</th>
        <th style="padding: 1em;">Possible Mitigations</th>
    </tr>
    <tr bgcolor="#ffffff">
        <td style="padding: 1em;">During assembly</td>
        <td style="padding: 1em;">Manufacturer tampering</td>
        <td rowspan="3" style="padding: 1em;">
            <ul>
            <li>PCB x-ray inspection</li>
            <li>[https://cercuits.com/transparent-pcb/ Translucent PCB substrate] for visual inspection</li>
            <li>Encase PCB in glitter epoxy</li>
            <li>[https://www.mcmaster.com/tamper-evident-markers Tamper-evident paint]
            <li>Apply heat-sensitive paint to PCB (detects soldering)</li>
            <li>Vacuum-sealed enclosure with onetime-use pressure sensor</li>
            <li>Serial numbers in ROM on ICs</li>
            <li>Use common enough chip to reduce chance of silicon tampering</li>
            <li>Pogo board testing</li>
            </ul>
        </td>
    </tr>
    <tr bgcolor="#ffffff">
        <td rowspan="3" style="padding: 1em;">During transit to end-user</td>
        <td style="padding: 1em;">Vendor/reseller tampering</td>
    </tr>
    <tr bgcolor="#ffffff">
        <td rowspan="2" style="padding: 1em;">Mail interdiction</td>
    </tr>
    <tr bgcolor="#ffffff">
        <td style="padding: 1em;">Embed device in another device</td>
    </tr>
    <tr bgcolor="#ffffff">
        <td rowspan="3" style="padding: 1em;">At customer's install site</td>
        <td rowspan="2" style="padding: 1em;">Evil user tampering</td>
        <td style="padding: 1em;">Live system intrusion detection (examples??)</td>
    </tr>
    <tr bgcolor="#ffffff">
        <td style="padding: 1em;">Routine pogo board testing</td>
    </tr>
    <tr bgcolor="#ffffff">
        <td style="padding: 1em;">Passive monitoring, e.g.,
            <ul>
                <li>audio/video monitoring of keypresses</li>
                <li>RF leakage analysis</li>
                <li>power analysis</li>
            </ul>
        </td>
        <td style="padding: 1em;">???</td>
    </tr>
</table>
<h3>March 27, 2021</h3>
<p>Still looking for a non-BGA ARM processor that contains two Ethernet PHYs and has 0.5mm or greater lead pitch (for [https://www.voltera.io Voltera V-1] PCB production. We are now considering three options for design:</p>
* '''(A)''' Non-BGA ARM processor on a homemade mainboard ''[simplest but limited to non-gigabit speeds; may require SW porting]''
* '''(B)''' BGA ARM processor on a fabbed carrier board attached to a homemade mainboard ''[more complicated but higher performance and easier to source; may avoid SW porting]''
** E.g., [https://www.nxp.com/products/processors-and-microcontrollers/arm-processors/i-mx-applications-processors/i-mx-6-processors/i-mx-6solox-processors-heterogeneous-processing-with-arm-cortex-a9-and-cortex-m4-cores:i.MX6SX NXP i.MX6SoloX]
** Unpopulated carrier board will need to be examined, probably via x-ray, after production
** Carrier board reflow is still done by us
** Still need to look for more ways to increase provenance
* '''(C)''' Use an existing open source BGA-based project (e.g., [https://www.kosagi.com/w/index.php?title=Novena_Main_Page Kosagi Novena]) and modify it for trusted manufacturing ''[complexity likely between (A) and (B)]''
** Will still likely need a BGA carrier board from (B) but may reduce R&D costs of the project
<p><strong>Infrastructure</strong></p>
<p>We’ll be setting up an IRC server in parallel to testing local deployment of the [https://element.io Element] secure chat system.</p>
<h3>March 20, 2021</h3>
<h3>March 20, 2021</h3>
<p>In order to make forward progress on the hardware, we will choose OpenBSD as the initial operating system and an ARM-based architecture for the CPU. Users will be able to use their own ARM-compatible OS if they choose, including Linux and Plan9 (if software support is added).</p>
<p>In order to make forward progress on the hardware, we will choose OpenBSD as the initial operating system and an ARM-based architecture for the CPU. Users will be able to use their own ARM-compatible OS if they choose, including Linux and Plan9 (if software support is added).</p>
<p>We will first create a Software-Intent Proof of Concept (SIPoC), which will be an OpenBSD-based router running on a commercially available SBC. We may then want, as a Proof of Concept (PoC), to port the software stack to the Common Networks nodes at Noisebridge and begin deploying them across sites.</p>
<p>We will first create a Software-Intent Proof of Concept (SIPoC), which will be an OpenBSD-based router/firewall running on a commercially available SBC. We may then want, as a Proof of Concept (PoC), to port the software stack to the [[COR1A|Common Networks nodes at Noisebridge]] and begin deploying them across sites.</p>
<p>The first version of our custom hardware should be amendable to DIY manufacturing. This means no BGA parts. This severely restricts the list of CPUs/SoCs to older and lower-performance chips, limiting our capability to 100MBps initially. We'll expand this list as we discover more:</p>
<p>The first version of our custom hardware should be amendable to DIY manufacturing. This means no BGA parts. This severely restricts the list of CPUs/SoCs to older and lower-performance chips, limiting our capability to 100MBps initially. We'll expand this list as we discover more:</p>
<h4>List of non-BGA ARM SoCs</h4>
<h4>List of non-BGA ARM SoCs</h4>
* Current leading candidate: [https://www.nuvoton.com/export/resource-files/DS_NUC980_Series_EN_Rev1.11.pdf Nuvoton NUC980DR61Y] - 64-eLQFP 0.5mm
* <s>Current leading candidate: [https://www.nuvoton.com/export/resource-files/DS_NUC980_Series_EN_Rev1.11.pdf Nuvoton NUC980DR61Y] - 64-eLQFP 0.5mm</s>
** Pros: Low pin count, has 10/100 PHY, 64 MB SiP RAM, HW crypto, relatively cheap (~$10 low qty)
** <s>Pros: Low pin count, has 10/100 PHY, 64 MB SiP RAM, HW crypto, relatively cheap (~$10 low qty)</s>
** Cons: No native OpenBSD/NetBSD support, no video out, RAM may be tight, 0.5mm pitch is tough with Voltera, no USB 1 port (USB 2 needs 1 GHz scope for debugging)  
** <s>Cons: No native OpenBSD/NetBSD support, no video out, RAM may be tight, 0.5mm pitch is tough with Voltera, no USB 1 port (USB 2 needs 1 GHz scope for debugging)</s>
** [https://github.com/OpenNuvoton/NUC980_NonOS_BSP Open source sample code]
** <s>[https://github.com/OpenNuvoton/NUC980_NonOS_BSP Open source sample code]</s>
** [https://www.nuvoton.com/products/microprocessors/arm9-mpus/nuc980-industrial-control-iot-series/nuc980dr61yc/?group=Document&tab=2 Reference schematics & design docs]
** <s>[https://www.nuvoton.com/products/microprocessors/arm9-mpus/nuc980-industrial-control-iot-series/nuc980dr61yc/?group=Document&tab=2 Reference schematics & design docs]</s>
** Dealbreaker: Only one PHY on 0.5mm package
* [https://www.ti.com/document-viewer/AM1705/datasheet TI AM1705] - 176-PQFP 0.5mm
* [https://www.ti.com/document-viewer/AM1705/datasheet TI AM1705] - 176-PQFP 0.5mm
* [https://www.nxp.com/docs/en/fact-sheet/IMX233FS.pdf NXP i.MX233] - 128-LQFP 0.4mm
* [https://www.nxp.com/docs/en/fact-sheet/IMX233FS.pdf NXP i.MX233] - 128-LQFP 0.4mm
Line 40: Line 227:


<h2>Tentative Project Stages</h2>
<h2>Tentative Project Stages</h2>
* SIPoC: OpenBSD router on commercial SBC
* SIPoC: OpenBSD router/firewall on commercial SBC
* PoC: SW stack on Common Networks
* PoC: SW stack on Common Networks
* Proto 1 build: Low-speed (10/100 Mbps) DIY version
* Proto 1 build: Low-speed (10/100 Mbps) DIY version
Line 46: Line 233:


<h2>Possible Design Choices for Future Versions</h2>
<h2>Possible Design Choices for Future Versions</h2>
* CPU
* '''CPU'''
** ARM/ARM64 SoC
** ARM/ARM64 SoC
** RISC-V SoC
** RISC-V SoC
** FPGA
** FPGA
** Specifically no Intel/compatible architectures due to poor security record
** Specifically no Intel/compatible architectures due to poor security record
* OS / Application Code
* '''OS / Application Code'''
** OpenBSD
** OpenBSD
** Qubes
** Qubes
Line 57: Line 244:
** Plan9
** Plan9
** Custom FPGA code
** Custom FPGA code
*Trusted manufacturers
* '''Trusted manufacturers'''
** [https://www.sfcircuits.com San Francisco Circuits]?
** [https://www.sfcircuits.com San Francisco Circuits]?
** [https://www.circuitlaunch.com CircuitLaunch]?
** [https://www.circuitlaunch.com CircuitLaunch]?
** [https://bayareacircuits.com Bay Area Circuits]?


<h2>Open Questions</h2>
<h2>Open Questions</h2>
* How to offer root-of-trust guarantees to non-DIY customers
* How to offer root-of-trust guarantees to non-DIY customers


<h2>Meetings</h2>
<h2>Links to Resources</h2>
<p>We are currently (as of March 2021) meeting every Saturday at 14:00 PT (GMT-8) on the [https://jitsi.noisebridge.io/ohsnap Noisebridge Jitsi video platform].</p>
* '''Related projects'''
** [https://www.kosagi.com/w/index.php?title=Novena_Main_Page Kosagi Novena] - bunnie Huang's open hardware ARM SBC with OpenBSD support and dual Ethernet
** [https://www.turris.com/en/mox/overview Turris MOX] - open source computing & networking ecosystem built around the Marvell Armada 3720
** [https://opencircuits.com/index.php?title=Linuxstamp Linuxstamp] – open source non-BGA ARM9 Linux-capable SBC with 10/100 Ethernet on a 2-layer board
** [https://ethernetfmc.com EthernetFMC] - open hardware GigE transceiver with RGMII interface based on [https://www.marvell.com/content/dam/marvell/en/public-collateral/transceivers/marvell-phys-transceivers-alaska-88e151x-datasheet.pdf Marvell 88E1510 chip]
** [http://www.ethernut.de/en/hardware/enut5/index.html Ethernut 5] - open hardware ARM9 computer (Linux but no OpenBSD)
** [https://www.powerpc-notebook.org/en/ Open Hardware PowerPC laptop] - FOSS POWER9 PC
** [https://cloud.google.com/blog/products/identity-security/titan-in-depth-security-in-plaintext Google Titan] - open source secure boot solution
** Deprecated projects
*** [http://securityrouter.org/wiki/Main_Page SecurityRouter] - OpenBSD distro for network management
* '''Other OpenBSD-capable boards'''
** [https://www.raspberrypi.org Raspberry Pi]
** Pine64 [https://www.pine64.org/pinebook-pro/ PineBookPro] and [https://www.pine64.org/rockpro64/ RockPro64]
** [https://www.friendlyarm.com/index.php?route=product/product&product_id=284 NanoPi R4S]
* '''Open source / custom silicon'''
** [https://libre-soc.org Libre-SOC] - open source silicon SoC
** [https://www.fossi-foundation.org/projects FOSSi] - Free and Open Source Silicon Foundation
** [https://www.skywatertechnology.com Skywater] - low-cost foundry with 180nm and 130nm nodes
*** [https://github.com/google/skywater-pdk Skywater PDK]
** [https://efabless.com Efabless] - low-cost custom silicon manufacturing
** [http://parallel.princeton.edu/openpiton/ Princeton OpenPiton] - open source research microprocessor
** [https://www.youtube.com/watch?v=EczW2IWdnOM Tim 'mithro' Ansell] gives an overview of open source silicon tools
* '''Software & firmware'''
** [https://libreboot.org Libreboot] - freedom-respecting boot firmware
** [http://www.netbsd.org/docs/kernel/porting_netbsd_arm_soc.html Porting NetBSD to ARM926EJ-S] - article on adding support to NetBSD for the same ARM9 core that the Nuvoton NUC980 uses
* '''Other architectures'''
** RISC-V:
*** OpenBSD on RISC-V: [https://github.com/MengshiLi/openbsd-riscv-notes/blob/master/Porting_OpenBSD_to_RISCV_FinalReport.pdf Porting OpenBSD to RISC-V ISA] (PDF), [https://github.com/MengshiLi/openbsd-riscv-notes GitHub repo]
*** [https://www.sifive.com SiFive] - custom RISC-V manufacturer in the South Bay
*** [https://openfive.com/custom-silicon/ OpenFive] - custom RISC-V manufacturer in the South Bay
** [https://millcomputing.com Mill Computing] - novel CPU design in the North Bay
** [https://www-50.ibm.com/systems/power/openpower/welcome.xhtml POWER9] – open source ISA that's [https://www.openbsd.org/powerpc64.html supported by OpenBSD]
*** [https://github.com/open-power OpenPOWER GitHub repo]
*** [https://secure.raptorcs.com/content/BK1SD1/intro.html Raptor Blackbird Secure Desktop] - FOSS server-class POWER9 computer
**** [https://www.osnews.com/story/133093/review-blackbird-secure-desktop-a-fully-open-source-modern-power9-workstation-without-any-proprietary-code/ Review]
* '''Communication & collaboration platforms'''
** [https://element.io Element] / [https://matrix.org Matrix] - open source, distributed, secure chat
** [https://github.com/wireapp Wire] - open source, secure chat
** [https://scuttlebutt.nz Scuttlebutt] - open source, distributed, secure social network
** [https://gitter.im Gitter] - open source chat
* '''Hardware manufacturers'''
** [https://www.sfcircuits.com San Francisco Circuits]
** [https://www.circuitlaunch.com CircuitLaunch]
** [https://oshpark.com OSHPark]
** [https://bayareacircuits.com Bay Area Circuits]
** [https://emsolutionstech.com EM Solutions] - PCB assembly
* '''Design tools'''
** [https://kicad.org KiCAD] - schematic capture and PCB layout
* '''Hardware anti-tampering'''
** [https://cercuits.com/transparent-pcb/ Transparent PCB] manufacturer
** [https://www.mcmaster.com/tamper-evident-markers Tamper evident markers]
** Wikipedia article on [https://en.wikipedia.org/wiki/Tamper-evident_technology Tamper Evident Technology]
** [https://media.ccc.de/v/36c3-10690-open_source_is_insufficient_to_solve_trust_problems_in_hardware#t=2848 "Open Source is insufficient to solve trust problems in hardware"] by Bunnie Huang (video presentation)
** [https://dustidentity.com DustIdentity] embedded diamond nanocrystals for system ID
** [https://spectrum.ieee.org/tech-talk/computing/hardware/chaos-programmable-chips-secure PUFs] (Physically Unclonable Functions) created in silicon on the IC wafer
*** [https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=9380284 Paper] describing principle
** [https://sharps.org/wp-content/uploads/BECKER-CHES.pdf Stealthy Dopant-Level Hardware Trojans]

Revision as of 10:33, 10 September 2021

Open Hardware for Secure Networks And Privacy (OHSNAP)

This is the project page for OHSNAP, an open-source platform for building secure networks with a known hardware root of trust.

Motivation

Virtually all commercially available networking equipment is proprietary and closed-source and cannot be independently verified to be free of malware. There have been documented cases of attackers – sometimes entire nation-states – physically modifying networking equipment and networkable devices in order to exfiltrate data and/or command & control otherwise-trusted systems. This leaves the average individual with little choice but to hope that their home network consists of and is secured by devices which do not phone home or contain other backdoors. Such a situation breaks the guarantee that the user's data and devices remain their sovereign property and instead places control into the hands of manufacturers and governments.

The goal of this project is to produce completely open designs and implementations for critical network infrastructure with auditable roots of trust. By making the hardware design, manufacturing process, and firmware and software stacks fully auditable, it allows users to inspect the entire end-to-end flow of their data and to directly control some or all of the fabrication of the device in order to establish positive provenance.

Device Summary

OHSNAP Firewall platform block diagram
OHSNAP Server platform block diagram

OHSNAP is an exploratory platform for secure hardware. The first two OHSNAP devices will be:

  • OHSNAP Firewall: An embedded computer running an open source firmware and OS. It will expose at least two gigabit Ethernet ports and is intended to be a trusted checkpoint between two untrusted networks.
  • OHSNAP Server: A small, low-power, single-board computer with a single 10/100 Ethernet port. May be used as a server, client, or IoT device.

Design Goals

  • No closed-source firmware or software allowed anywhere in the stack
  • Implementation must be independently reproducible by third parties
  • Factory-made PCBs must be avoided if possible or physically produced in the USA when necessary
  • Components should be as supplier-diversified as possible
  • Every phase of manufacturing must be either tamper-proof or tamper-evident
  • Fewer pieces = smaller attack surface

Meetings

We are currently meeting every Thursday at 16:30 PT (GMT-8) on the Noisebridge Jitsi video platform.

Status

OHSNAP Server engineering timeline

August 10, 2021

  • Work on Nuvoton dev board PCB layout is in progress; expected checkpoint in September
  • Identified current main need: We need to grow the team to help solve major root-of-trust questions
    • Need an easily digestible web presence (like a one-pager with an easy URL) to quickly show prospective collaborators
      • We need a web designer to help with this!
    • Move to a more distributed collaboration platform (Element/Matrix)
    • Get more visuals (diagrams, photos) of plan and works-in-progress
    • Continue to reach out to potential contributors and allies
      • Can YOU help?
  • Discussed alternatives to SiFive/OpenFive + related projects


August 3, 2021

Meeting notes with SiFive / OpenFive:

  • Can they do it? Yes: OpenFive has the capabilities to package both custom and standard cores to customer specs and can easily put the UC74-MC into a QFP package.
  • Costs for any ASIC work are divided between IP and manufacturing.
    • Manufacturing cost scales inversely with process node.
      • 28nm process is roughly $1M for masks and another $1M for tapeout + packaging + validation. This is for a typical/minimum order quantity of 10,000.
        • For comparison, 5nm processes are on the order of $20M+.
      • The maximum process node is determined by design requirements (particularly max frequency, die size, power consumption, thermal dissipation). For OHSNAP Firewall our estimated lower frequency is ~700 MHz. This most likely means a maximum of 28nm process.
      • Re-spins are often necessary for most clients due to silicon bugs. For 28nm, this adds a typical $150-200k for a 3 layer respin.
    • IP cost depends on what core we use and what options we select. The stock UC74-MC core that we were considering is about ~$1M. Additional features drive the price up. Switching from -MC (multicore) to single core only saves a modest amount of money.
      • They offered a lower-end alternative to the U7-series: The U5 series. It is also Linux capable but has a shorter pipeline and so isn’t as performant. Typical IP is $500k.
  • Time: Typical time between submitting final RTL designs and start of tapeout is 1 year. This includes 1 week to generate masks, 10-12 weeks to process the silicon (for 28nm), postprocessing time (may not be necessary for QFP packages), die assembly at the package assembly site, and testing. Add another 4-5 months until we can receive engineering samples. Total time for the entire process from start to ramp is 1.5-2 years.
  • Tamper evidence / manufacturing control:
    • OpenFive uses two foundries: TSMC and GlobalFoundries. Our requirement for domestic manufacturing eliminates TSMC and 6 out of the 9 GlobalFoundries sites, but the remaining 3 are viable candidates. According to this article, two of those can hit 28nm and both are in New York.
    • Three main arenas to consider: Silicon manufacturing, packaging + assembly, and testing.
    • OpenFive transports the products themselves.
    • They would be open to us installing an observer to monitor production (for a fee).
    • They would be open to providing us the results of Layout-Versus-Schematic (LVS) tests.
  • Development cycle
    • 1. Pre-silicon: Emulation
      • QEMU
      • RTL
      • FPGA Bitstream
      • System C
    • 2. Dev board
      • For U5/U7 series chips: HiFive Unmatched
      • Do bulk of development while waiting for engineering samples
    • 3. Engineering samples
      • Finish main development
      • Debug debug debug
      • Possibly re-spin Rev B silicon
  • They briefly mentioned SiFive Shield and WorldGuard, two architectural technologies for enhanced silicon security. While these don’t solve our problem of trusted manufacturing, they are definitely useful layers to add to our secure ecosystem.
  • Summary: They seemed ready to engage with us but the cost is the big question. They were impressed by our mission and our passion and wanted to know more about the device, its use cases, and our business model. We told them that they are our Plan A and we really want to work with them but their cost estimates were an order of magnitude greater than we anticipated so we have to go back to the drawing board.

July 27, 2021

Preparing for meeting with SiFive to discuss custom packaging of the UC74-MC RISC-V core.

July 9, 2021

Explored RISC-V option. Many advantages:

  • We can reduce the attack surface by including only the silicon IP blocks that we need
  • Eliminates the requirement for outsourcing PCB manufacture
  • One chip can serve as the base platform for all OHSNAP projects (firewall, server, TFC, etc.) as well as being adoptable by third parties
  • Potentially improves best-case root-of-trust guarantee (but this requires a lot of thought and care)
  • Extends the idea of Open Hardware beyond traditional limits!

Main barriers:

  • High cost of entry
  • Need vendor buy-in for enhanced tamper evident manufacturing practices
  • Significant development effort

We will deliberate and vote on this. If in consensus, next step is to engage vendor(s) and start understanding the height of the above barriers.

July 2, 2021

Examination of the OpenBSD source and mailing list archives shows that support for pre-v7 ARM architectures was intentionally removed from the OS. According to this table, the NUC980's ARM926EJ core is ARMv5-based and therefore unsupported.

Adding support for ARMv5 to OpenBSD is out of scope for this project and we're not aware of any plans by the OpenBSD developers to do so either. This means that we can't use the NUC980 with OpenBSD for the OHSNAP Server and leaves us with these choices:

  1. Pause the OHSNAP Server project and continue to focus on OHSNAP Firewall
  2. Resume searching for OpenBSD-compatible non-BGA chips that are similar to the Nuvoton (unlikely to exist on the market, as we have been searching for a while and chips like this tend to be older designs and therefore unlikely to be supported)
  3. Migrate to a different OS which has ARM926EJ support, such as NetBSD or Linux. Both are considered less secure than OpenBSD, the latter unacceptably so.
  4. With newly added and quickly expanding RISC-V support in OpenBSD, we may want to consider that architecture more seriously. One key advantage of open source silicon is that we can create a custom, high-performance chip and put it in a QFP package with only the pins exposed that we would need. Next steps would be getting ballpark estimates of taping out our own silicon from local vendors.

June 25, 2021

  • Created v0.1 schematic for NUC980-based dev board. Will first be used with bare metal firmware.
  • Ordered samples of NUC980DR61Y and the Nuvoton Chili dev board. The latter will be used to test firmware and Linux and to begin the OpenBSD port.
  • Found article on porting NetBSD to ARM926EJ-S, the same core as in the Nuvoton chip.

June 18, 2021

We've settled on a roadmap:

  1. Create a fully homebuilt, low-performance 10/100 single Ethernet board that is capable of running OpenBSD ("Option A" from March 27, 2021). This approach will:
    1. Provide us with hands-on experience in designing, building, debugging our own hardware.
    2. Give us a testbed for experimenting with secure manufacturing practices.
    3. Require us to learn how to port OpenBSD to a new chip and board.
    4. Provide a ready-to-use reference design for other, related secure manufacturing projects at Noisebridge and beyond.
    5. Produce a physical deliverable that will create buy-in and grow our ecosystem.
  2. In parallel, we will continue to identify and engage potential vendors for high-performance designs which require securely outsourcing manufacturing.

The first low-performance design will be built around the Nuvoton NUC980DR61Y (see March 20, 2021). This will come in stages:

  1. HWPOC (Hardware Proof Of Concept): A minimal board designed to allow creating bare metal firmware to test basic design and in-house manufacturing on the Voltera PCB printer. We may attempt to boot Linux from this board, as there is kernel support for it and the design will be derived from the Nuvoton Chili dev board.
  2. SWPOC (Software Proof of Concept): Use the Nuvoton Chili board to port OpenBSD to the target chip.
  3. Proto: Boot OpenBSD from the minimal in-house board.
  4. EVT: First in-house board featuring Ethernet and all planned features. Still contains debugging connections.
  5. DVT: Finalized PCB. Begin testing low-qty mass production using pick & place machine.
  6. PVT: Final production units. Goes on to ramp build.

May 8, 2021

  • Continued exploration of threat vectors and possible mitigations.
  • More links added to resource list.
  • Setting up Tinfoil Chat as a realtime collaboration platform and exploration of secure methodologies.
  • Setting up OpenBSD on various SBCs and virtual machines to evaluate its fitness for use.

April 3, 2021

Identified three regimes for threat models, some concrete examples, and some hypothetical mitigations:

Threat Regime Example Scenario Possible Mitigations
During assembly Manufacturer tampering
  • PCB x-ray inspection
  • Translucent PCB substrate for visual inspection
  • Encase PCB in glitter epoxy
  • Tamper-evident paint
  • Apply heat-sensitive paint to PCB (detects soldering)
  • Vacuum-sealed enclosure with onetime-use pressure sensor
  • Serial numbers in ROM on ICs
  • Use common enough chip to reduce chance of silicon tampering
  • Pogo board testing
During transit to end-user Vendor/reseller tampering
Mail interdiction
Embed device in another device
At customer's install site Evil user tampering Live system intrusion detection (examples??)
Routine pogo board testing
Passive monitoring, e.g.,
  • audio/video monitoring of keypresses
  • RF leakage analysis
  • power analysis
???

March 27, 2021

Still looking for a non-BGA ARM processor that contains two Ethernet PHYs and has 0.5mm or greater lead pitch (for Voltera V-1 PCB production. We are now considering three options for design:

  • (A) Non-BGA ARM processor on a homemade mainboard [simplest but limited to non-gigabit speeds; may require SW porting]
  • (B) BGA ARM processor on a fabbed carrier board attached to a homemade mainboard [more complicated but higher performance and easier to source; may avoid SW porting]
    • E.g., NXP i.MX6SoloX
    • Unpopulated carrier board will need to be examined, probably via x-ray, after production
    • Carrier board reflow is still done by us
    • Still need to look for more ways to increase provenance
  • (C) Use an existing open source BGA-based project (e.g., Kosagi Novena) and modify it for trusted manufacturing [complexity likely between (A) and (B)]
    • Will still likely need a BGA carrier board from (B) but may reduce R&D costs of the project

Infrastructure

We’ll be setting up an IRC server in parallel to testing local deployment of the Element secure chat system.

March 20, 2021

In order to make forward progress on the hardware, we will choose OpenBSD as the initial operating system and an ARM-based architecture for the CPU. Users will be able to use their own ARM-compatible OS if they choose, including Linux and Plan9 (if software support is added).

We will first create a Software-Intent Proof of Concept (SIPoC), which will be an OpenBSD-based router/firewall running on a commercially available SBC. We may then want, as a Proof of Concept (PoC), to port the software stack to the Common Networks nodes at Noisebridge and begin deploying them across sites.

The first version of our custom hardware should be amendable to DIY manufacturing. This means no BGA parts. This severely restricts the list of CPUs/SoCs to older and lower-performance chips, limiting our capability to 100MBps initially. We'll expand this list as we discover more:

List of non-BGA ARM SoCs

March 13, 2021

  • Looking for secure communications platform for project collaboration. Element / Matrix look promising.
  • Possibly partner with CircuitLaunch for local hardware builds?

March 6, 2021

Initial meeting. Discussed range of HW/SW design choices.

Tentative Project Stages

  • SIPoC: OpenBSD router/firewall on commercial SBC
  • PoC: SW stack on Common Networks
  • Proto 1 build: Low-speed (10/100 Mbps) DIY version
  • Full build: 1 Gbps

Possible Design Choices for Future Versions

  • CPU
    • ARM/ARM64 SoC
    • RISC-V SoC
    • FPGA
    • Specifically no Intel/compatible architectures due to poor security record
  • OS / Application Code
    • OpenBSD
    • Qubes
    • Alpine Linux
    • Plan9
    • Custom FPGA code
  • Trusted manufacturers

Open Questions

  • How to offer root-of-trust guarantees to non-DIY customers

Links to Resources