OHSNAP: Difference between revisions

From Noisebridge
Jump to navigation Jump to search
No edit summary
(Added June 2022 status. Cleaned up page a bit.)
(23 intermediate revisions by 3 users not shown)
Line 1: Line 1:
{{security}}
{{box}}[[File:OHSNAP before after.png|200px|right]]
'''OHSNAP''' (Open Hardware for Secure Networks And Privacy), is an open-source platform for building secure networks with a known hardware root of trust.
<br>'''MAINTAINERS:''' [[User:Pion]], [[User:Lizzard]]. [[User:James]]
{{boxend}}
<h1>Open Hardware for Secure Networks And Privacy (OHSNAP)</h1>
<h1>Open Hardware for Secure Networks And Privacy (OHSNAP)</h1>
<span id="before-after">[[File:OHSNAP_before_after.png|right|600px|alt=Before and After OHSNAP Firewall in your network]]</span>


This is the project page for OHSNAP, an open-source platform for building secure networks with a known hardware root of trust.
This is the project page for OHSNAP, an open-source platform for building secure networks with a known hardware root of trust.


<h2>Motivation</h2>
<h2>Motivation</h2>
<p>Virtually all commercially-available networking equipment is proprietary and closed-source and cannot be independently verified to be free of malware. There have been documented cases of attackers – [https://www.bloomberg.com/news/features/2018-10-04/the-big-hack-how-china-used-a-tiny-chip-to-infiltrate-america-s-top-companies sometimes entire nation-states] – physically modifying networking equipment and networkable devices in order to exfiltrate data and/or command & control otherwise-trusted systems. This leaves the average individual with little choice but to hope that their home network consists of and is secured by devices which do not phone home or contain other backdoors. Such a situation breaks the guarantee that the user's data and devices remain their sovereign property and instead places control into the hands of manufacturers and governments.</p>
<p>Virtually all commercially available networking equipment is proprietary and closed-source and cannot be independently verified to be free of malware. There have been documented cases of attackers – [https://www.bloomberg.com/news/features/2018-10-04/the-big-hack-how-china-used-a-tiny-chip-to-infiltrate-america-s-top-companies sometimes entire nation-states] – physically modifying networking equipment and networkable devices in order to exfiltrate data and/or command & control otherwise-trusted systems. This leaves the average individual with little choice but to hope that their home network consists of and is secured by devices which do not phone home or contain other backdoors. Such a situation breaks the guarantee that the user's data and devices remain their sovereign property and instead places control into the hands of manufacturers and governments.</p>
 
<p>The goal of this project is to produce completely open designs and implementations for critical network infrastructure with auditable roots of trust. By making the hardware design, manufacturing process, and firmware and software stacks fully auditable, it allows users to inspect the entire end-to-end flow of their data and to directly control some or all of the fabrication of the device in order to establish positive provenance.</p>


<p>The goal of this project is to produce completely open designs and implementations for critical network infrastructure with verifiable roots of trust. By making the hardware design, manufacturing process, and firmware and software stacks fully verifiable, it allows users to inspect the entire end-to-end flow of their data and to directly control some or all of the fabrication of the device in order to establish positive provenance.</p>


<h2>Device Summary</h2>
<h2>Device Summary</h2>
<span id="firewall-diagram">[[File:ohsnap_firewall_block_diagram_v0.11.png|right|600px|alt=OHSNAP Firewall platform block diagram]]</span>
<span id="server-diagram">[[File:ohsnap_server_block_diagram_v0.1.png|right|600px|alt=OHSNAP Server platform block diagram]]</span>
OHSNAP is an exploratory platform for secure hardware. The first two OHSNAP devices will be:
OHSNAP is an exploratory platform for secure hardware. The first two OHSNAP devices will be:
* OHSNAP Router: An embedded computer running an open source firmware and OS. It will expose at least two gigabit Ethernet ports.
* '''OHSNAP Firewall''': An embedded computer running an open source firmware and OS. It will expose at least two gigabit Ethernet ports and is intended to be a trusted checkpoint between two untrusted networks. The default software configuration will be OpenBSD running a custom connection monitoring, alerting, and filtering tool.
* OHSNAP Server: A small, low-power, single-board computer with a single 10/100 Ethernet port.
* '''OHSNAP Server''': A small, low-power, single-board computer with a single 10/100 Ethernet port. May be used as a server, client, or IoT device. Will run NetBSD.


<h2>Design Goals</h2>
<h2>Design Security Principles</h2>
* No closed-source firmware or software allowed anywhere in the stack
* No closed-source firmware or software allowed anywhere in the stack
* Implementation must be independently reproducible by third parties
* Implementation must be independently reproducible by third parties
* Factory-made PCBs must be physically produced in the USA
* Factory-made PCBs must be avoided if possible or physically produced in the USA when necessary
* Components should be as supplier-diversified as possible
* Components should be as supplier-diversified as possible
* Every phase of manufacturing must be either tamper-proof or tamper-evident
* Fewer pieces = smaller attack surface
<h2>Open Source Repository</h2>
All design documentation, schematics, firmware, and software will be available at [https://github.com/trustOHSNAP our GitHub repository].
<h2>Meetings</h2>
<p>We are currently meeting every '''Thursday at 16:30 PT''' (GMT-8) using the [https://jitsi.bench.supply/ohsnap Jitsi video platform]. '''If you can't connect:''' Sometimes our primary Jitsi instance goes down. If you're having connection issues, [https://meet.jit.si/ohsnap try our backup Jitsi].
<h2>How to Get Involved</h2>
The best way to get involved is to come to one of our weekly [[#Meetings|meetings]]. '''All forms of help and areas of expertise are welcome and needed!''' This is a crossfunctional, multidisciplinary project that requires people with all sorts of talents and interests to collaborate if it is to succeed.
In addition to graciously accepting general help and contributions, we have a current need for specific expertise in the following areas:
* Low-level software/OS engineering
* Firmware engineering
* Hardware engineering
* Materials science
* Cryptography
* Network security
* Web design
<h2>Open Questions</h2>
* How to offer root-of-trust guarantees to non-DIY customers?


<h2>Status</h2>
<h2>Status</h2>
<span id="server-timeline">[[File:ohsnap_server_timeline_v0.1.png|right|600px|alt=OHSNAP Server engineering timeline]]</span>
<h3>June 2022</h3>
* New software initiatives:
** Core functionality of [[#Device Summary|OHSNAP Firewall]] will require custom software on the firewall and optionally clients. This will also be a standalone software product that's useful outside of OHSNAP hardware.
*** Firewall software monitors and filters new connections to/from all downstream clients (hooks into [https://www.openbsd.org/faq/pf/ OpenBSD's pf])
*** Any new connection attempts (which haven't previously been preconfigured via <code>pf</code> rules) from client devices will generate an alert to one or more registered alerting devices and optionally on clients who want local first-responder rights
*** Users allow or deny the connection (either on a native client or on the alerting device) and optionally save their decision as a rule.
*** All connections can be logged and searched for later forensic discovery or trend monitoring.
** <code>oort</code> (OHSNAP OpenBSD Rapid Provisioning Tools): A toolkit that can automate the provisioning of multiple devices with OpenBSD. Useful for both speeding up our debug cycles and for final imaging of hardware deliverables. Will also be a standalone software product that's usable outside of OHSNAP hardware.
* [[#Device Summary|OHSNAP Server]] (with an appropriate UART LCD, Allow/Deny pushbuttons, and optional buzzer/vibration motor) can be used as a standalone alerting device for above.
* Created new top-level dev repositories ([https://github.com/trustOHSNAP/OHSNAP-Server server], [https://github.com/trustOHSNAP/OHSNAP-Firewall firewall], [https://github.com/trustOHSNAP/OHSNAP-Rapid-Provisioning-Tools oort]) along with a source control policy and workflow for the team.
* Separate new repo ([https://github.com/trustOHSNAP/docs docs]) for links and other useful documentation of our research
<h3>February 2022</h3>
* Continuing discussion on OHSNAP Firewall architecture
** May consider adopting [https://github.com/maqp/tfc TFC]-like separation between RGMII send & receive signals
*** Requires splitting design into 2-3 independent boards.
*** Could require some form of hardware authentication between boards. Key generated by using at time of firmware compilation & manual flashing.
*** User can optionally be required to add final Ethernet jacks manually, enhancing security
* Restarted working on OHSNAP Server hardware
** Continuing PoC board layout
** Minor schematic updates include updating to KiCAD 6, creating a pin mapping for tracking resource allocation in firmware, and adding an SD card module for bootloading.
** Following updates of [https://www.voltera.io Voltera] NOVA (desktop additive flexible electronics manufacturing) and [https://efabless.com Efabless] (open source custom silicon manufacturing) for possible use in project
* Added a [[#before-after|graphic]] showing OHSNAP Firewall as part of a user's network ecosystem.
<h3>November 11, 2021</h3>
Had some discussion on the OHSNAP Server boot process
* HW will have SPI flash, one SD card, and one USB port
* Primary bootloader will reside in SPI flash (may be updated through JTAG)
* Boot installer image or installed OS from SD card
* Installer will write OS to USB flash drive
* SW Development sequence:
** 1. Port NetBSD to NUC980
** 2. Write USB driver layer
** 3. Write SD card driver layer
Hardware To-Do: Need to add SD card
<h3>October 28, 2021</h3>
* Work has started on adding QEMU support for the NUC980 chip which we are using in OHSNAP Server. Development is currently on the [https://github.com/noisebridge/OHSNAP/tree/develop develop] branch.
<h3>August 10, 2021</h3>
* Work on Nuvoton dev board PCB layout is in progress; expected checkpoint in September
* Identified current main need: '''We need to grow the team to help solve major root-of-trust questions'''
** Need an easily digestible web presence (like a one-pager with an easy URL) to quickly show prospective collaborators
*** '''We need a web designer to help with this!'''
** Move to a more distributed collaboration platform (Element/Matrix)
** Get more visuals (diagrams, photos) of plan and works-in-progress
*** Added [[#firewall-diagram|block diagram for OHSNAP Firewall]]
** Continue to reach out to potential contributors and allies
*** '''Can YOU help?'''
* Discussed alternatives to SiFive/OpenFive + related projects
** [https://www.skywatertechnology.com Skywater] - low-cost foundry with 180nm and 130nm nodes
*** [https://github.com/google/skywater-pdk Skywater PDK]
** [https://efabless.com Efabless] is offering free (in conjunction with Google) and low-cost custom silicon
** [https://libre-soc.org Libre-SOC] - open source silicon SoC
** [http://parallel.princeton.edu/openpiton/ Princeton OpenPiton] - open source research microprocessor
** [https://www.youtube.com/watch?v=EczW2IWdnOM Tim 'mithro' Ansell] gives an overview of open source silicon tools
<h3>August 3, 2021</h3>
Meeting notes with [https://www.sifive.com SiFive] / [https://openfive.com OpenFive]:
* '''Can they do it?''' Yes: OpenFive has the capabilities to package both custom and standard cores to customer specs and can easily put the UC74-MC into a QFP package.
* '''Costs''' for ''any'' ASIC work are divided between IP and manufacturing.
** Manufacturing cost scales inversely with process node.
*** 28nm process is roughly $1M for masks and another $1M for tapeout + packaging + validation. This is for a typical/minimum order quantity of 10,000.
**** For comparison, 5nm processes are on the order of $20M+.
*** The maximum process node is determined by design requirements (particularly max frequency, die size, power consumption, thermal dissipation). For OHSNAP Firewall our estimated lower frequency is ~700 MHz. This most likely means a maximum of 28nm process.
*** Re-spins are often necessary for most clients due to silicon bugs. For 28nm, this adds a typical $150-200k for a 3 layer respin.
** IP cost depends on what core we use and what options we select. The stock [https://www.sifive.com/cores/u74-mc UC74-MC core] that we were considering is about ~$1M. Additional features drive the price up. Switching from -MC (multicore) to [https://www.sifive.com/cores/u74 single core] only saves a modest amount of money.
*** They offered a lower-end alternative to the U7-series: The [https://www.sifive.com/cores/u54 U5 series]. It is also Linux capable but has a shorter pipeline and so isn’t as performant. Typical IP is $500k.
* '''Time''': Typical time between submitting final RTL designs and start of tapeout is 1 year. This includes 1 week to generate masks, 10-12 weeks to process the silicon (for 28nm), postprocessing time (may not be necessary for QFP packages), die assembly at the package assembly site, and testing. Add another 4-5 months until we can receive engineering samples. Total time for the entire process from start to ramp is 1.5-2 years.
* '''Tamper evidence / manufacturing control''':
** OpenFive uses two foundries: TSMC and GlobalFoundries. Our requirement for domestic manufacturing eliminates TSMC and 6 out of the 9 GlobalFoundries sites, but the remaining 3 are viable candidates. According to [https://en.wikipedia.org/wiki/GlobalFoundries#Fabrication_plants this article], two of those can hit 28nm and both are in New York.
** Three main arenas to consider: Silicon manufacturing, packaging + assembly, and testing.
** OpenFive transports the products themselves.
** They would be open to us installing an observer to monitor production (for a fee).
** They would be open to providing us the results of Layout-Versus-Schematic (LVS) tests.
* '''Development cycle'''
** 1. Pre-silicon: Emulation
*** QEMU
*** RTL
*** FPGA Bitstream
*** System C
** 2. Dev board
*** For U5/U7 series chips: [https://www.sifive.com/boards/hifive-unmatched HiFive Unmatched]
*** Do bulk of development while waiting for engineering samples
** 3. Engineering samples
*** Finish main development
*** Debug debug debug
*** Possibly re-spin Rev B silicon
* They briefly mentioned [https://www.sifive.com/blog/sifive-shield-an-open-scalable-platform-architecture SiFive Shield and WorldGuard], two architectural technologies for enhanced silicon security. While these don’t solve our problem of trusted manufacturing, they are definitely useful layers to add to our secure ecosystem.
* '''Summary:''' They seemed ready to engage with us but the cost is the big question. They were impressed by our mission and our passion and wanted to know more about the device, its use cases, and our business model. We told them that they are our Plan A and we really want to work with them but their cost estimates were an order of magnitude greater than we anticipated so we have to go back to the drawing board.
<h3>July 27, 2021</h3>
Preparing for meeting with SiFive to discuss custom packaging of the UC74-MC RISC-V core.
<h3>July 9, 2021</h3>
<h3>July 9, 2021</h3>
Explored RISC-V option. Many advantages:
Explored RISC-V option. Many advantages:
* We can reduce the attack surface by including only the silicon IP blocks that we need
* We can reduce the attack surface by including only the silicon IP blocks that we need
* Eliminates the requirement for outsourcing PCB manufacture
* Eliminates the requirement for outsourcing PCB manufacture
* One chip can serve as the base platform for all OHSNAP projects (router, server, TFC, etc.) as well as being adoptable by third parties
* One chip can serve as the base platform for all OHSNAP projects (firewall, server, TFC, etc.) as well as being adoptable by third parties
* ''Potentially'' improves best-case root-of-trust guarantee (but this requires a lot of thought and care)
* ''Potentially'' improves best-case root-of-trust guarantee (but this requires a lot of thought and care)
* Extends the idea of Open Hardware beyond traditional limits!
* Extends the idea of Open Hardware beyond traditional limits!
Line 37: Line 166:


Adding support for ARMv5 to OpenBSD is out of scope for this project and we're not aware of any plans by the OpenBSD developers to do so either. This means that we can't use the NUC980 with OpenBSD for the OHSNAP Server and leaves us with these choices:
Adding support for ARMv5 to OpenBSD is out of scope for this project and we're not aware of any plans by the OpenBSD developers to do so either. This means that we can't use the NUC980 with OpenBSD for the OHSNAP Server and leaves us with these choices:
# Pause the OHSNAP Server project and continue to focus on OHSNAP Router
# Pause the OHSNAP Server project and continue to focus on OHSNAP Firewall
# Resume searching for OpenBSD-compatible non-BGA chips that are similar to the Nuvoton (unlikely to exist on the market, as we have been searching for a while and chips like this tend to be older designs and therefore unlikely to be supported)
# Resume searching for OpenBSD-compatible non-BGA chips that are similar to the Nuvoton (unlikely to exist on the market, as we have been searching for a while and chips like this tend to be older designs and therefore unlikely to be supported)
# Migrate to a different OS which has ARM926EJ support, such as NetBSD or Linux. Both are considered less secure than OpenBSD, the latter unacceptably so.
# Migrate to a different OS which has ARM926EJ support, such as NetBSD or Linux. Both are considered less secure than OpenBSD, the latter unacceptably so.
Line 43: Line 172:


<h3>June 25, 2021</h3>
<h3>June 25, 2021</h3>
* Created v0.1 schematic for NUC980-based dev board. Will first be used with bare metal firmware.
* Created [https://github.com/trustOHSNAP/OHSNAP-Server/tree/main/electrical v0.1 schematic for NUC980-based dev board]. Will first be used with bare metal firmware.
* Ordered samples of NUC980DR61Y and the Nuvoton Chili dev board. The latter will be used to test firmware and Linux and to begin the OpenBSD port.
* Ordered samples of NUC980DR61Y and the Nuvoton Chili dev board. The latter will be used to test firmware and Linux and to begin the OpenBSD port.
* Found [http://www.netbsd.org/docs/kernel/porting_netbsd_arm_soc.html article] on porting NetBSD to ARM926EJ-S, the same core as in the Nuvoton chip.
* Found [http://www.netbsd.org/docs/kernel/porting_netbsd_arm_soc.html article] on porting NetBSD to ARM926EJ-S, the same core as in the Nuvoton chip.
Line 141: Line 270:
<h3>March 20, 2021</h3>
<h3>March 20, 2021</h3>
<p>In order to make forward progress on the hardware, we will choose OpenBSD as the initial operating system and an ARM-based architecture for the CPU. Users will be able to use their own ARM-compatible OS if they choose, including Linux and Plan9 (if software support is added).</p>
<p>In order to make forward progress on the hardware, we will choose OpenBSD as the initial operating system and an ARM-based architecture for the CPU. Users will be able to use their own ARM-compatible OS if they choose, including Linux and Plan9 (if software support is added).</p>
<p>We will first create a Software-Intent Proof of Concept (SIPoC), which will be an OpenBSD-based router running on a commercially available SBC. We may then want, as a Proof of Concept (PoC), to port the software stack to the Common Networks nodes at Noisebridge and begin deploying them across sites.</p>
<p>We will first create a Software-Intent Proof of Concept (SIPoC), which will be an OpenBSD-based router/firewall running on a commercially available SBC. We may then want, as a Proof of Concept (PoC), to port the software stack to the [[COR1A|Common Networks nodes at Noisebridge]] and begin deploying them across sites.</p>
<p>The first version of our custom hardware should be amendable to DIY manufacturing. This means no BGA parts. This severely restricts the list of CPUs/SoCs to older and lower-performance chips, limiting our capability to 100MBps initially. We'll expand this list as we discover more:</p>
<p>The first version of our custom hardware should be amendable to DIY manufacturing. This means no BGA parts. This severely restricts the list of CPUs/SoCs to older and lower-performance chips, limiting our capability to 100MBps initially. We'll expand this list as we discover more:</p>
<h4>List of non-BGA ARM SoCs</h4>
<h4>List of non-BGA ARM SoCs</h4>
Line 159: Line 288:


<h3>March 6, 2021</h3>
<h3>March 6, 2021</h3>
Initial meeting. Discussed range of HW/SW design choices.
* Initial meeting. Discussed range of HW/SW design choices.
 
* Tentative Project Stages
<h2>Tentative Project Stages</h2>
** SIPoC: OpenBSD router/firewall on commercial SBC
* SIPoC: OpenBSD router on commercial SBC
** PoC: SW stack on Common Networks
* PoC: SW stack on Common Networks
** Proto 1 build: Low-speed (10/100 Mbps) DIY version
* Proto 1 build: Low-speed (10/100 Mbps) DIY version
** Full build: 1 Gbps
* Full build: 1 Gbps
* Explored possible design choices for future versions
 
** '''CPU'''
<h2>Possible Design Choices for Future Versions</h2>
*** ARM/ARM64 SoC
* '''CPU'''
*** RISC-V SoC
** ARM/ARM64 SoC
*** FPGA
** RISC-V SoC
*** Specifically no Intel/compatible architectures due to poor security record
** FPGA
** '''OS / Application Code'''
** Specifically no Intel/compatible architectures due to poor security record
*** OpenBSD
* '''OS / Application Code'''
*** Qubes
** OpenBSD
*** Alpine Linux
** Qubes
*** Plan9
** Alpine Linux
*** Custom FPGA code
** Plan9
** '''Trusted manufacturers'''
** Custom FPGA code
*** [https://www.sfcircuits.com San Francisco Circuits]?
* '''Trusted manufacturers'''
*** [https://www.circuitlaunch.com CircuitLaunch]?
** [https://www.sfcircuits.com San Francisco Circuits]?
*** [https://bayareacircuits.com Bay Area Circuits]?
** [https://www.circuitlaunch.com CircuitLaunch]?
** [https://bayareacircuits.com Bay Area Circuits]?
 
<h2>Open Questions</h2>
* How to offer root-of-trust guarantees to non-DIY customers
 
<h2>Meetings</h2>
<p>We are currently meeting every Tuesday at 16:30 PT (GMT-8) on the [https://jitsi.noisebridge.io/ohsnap Noisebridge Jitsi video platform].</p>


<h2>Links to Resources</h2>
<h2>Links to Resources</h2>
* '''Related projects'''
<h3>[https://github.com/trustOHSNAP/docs OHSNAP Internal Documentation Repository]</h3>
** [https://www.kosagi.com/w/index.php?title=Novena_Main_Page Kosagi Novena] - bunnie Huang's open hardware ARM SBC with OpenBSD support and dual Ethernet
Most new links will be added here: '''[https://github.com/trustOHSNAP/docs docs]'''
** [https://www.turris.com/en/mox/overview Turris MOX] - open source computing & networking ecosystem built around the Marvell Armada 3720
<h3>Related projects</h3>
** [https://opencircuits.com/index.php?title=Linuxstamp Linuxstamp] – open source non-BGA ARM9 Linux-capable SBC with 10/100 Ethernet on a 2-layer board
* [https://www.kosagi.com/w/index.php?title=Novena_Main_Page Kosagi Novena] - bunnie Huang's open hardware ARM SBC with OpenBSD support and dual Ethernet
** [https://ethernetfmc.com EthernetFMC] - open hardware GigE transceiver with RGMII interface based on [https://www.marvell.com/content/dam/marvell/en/public-collateral/transceivers/marvell-phys-transceivers-alaska-88e151x-datasheet.pdf Marvell 88E1510 chip]
* [https://www.turris.com/en/mox/overview Turris MOX] - open source computing & networking ecosystem built around the Marvell Armada 3720
** Deprecated projects
* [https://opencircuits.com/index.php?title=Linuxstamp Linuxstamp] – open source non-BGA ARM9 Linux-capable SBC with 10/100 Ethernet on a 2-layer board
*** [http://securityrouter.org/wiki/Main_Page SecurityRouter] - OpenBSD distro for network management
* [https://ethernetfmc.com EthernetFMC] - open hardware GigE transceiver with RGMII interface based on [https://www.marvell.com/content/dam/marvell/en/public-collateral/transceivers/marvell-phys-transceivers-alaska-88e151x-datasheet.pdf Marvell 88E1510 chip]
* '''Other OpenBSD-capable boards'''
* [http://www.ethernut.de/en/hardware/enut5/index.html Ethernut 5] - open hardware ARM9 computer (Linux but no OpenBSD)
** [https://www.raspberrypi.org Raspberry Pi]
* [https://www.powerpc-notebook.org/en/ Open Hardware PowerPC laptop] - FOSS POWER9 PC
** Pine64 [https://www.pine64.org/pinebook-pro/ PineBookPro] and [https://www.pine64.org/rockpro64/ RockPro64]
* [https://cloud.google.com/blog/products/identity-security/titan-in-depth-security-in-plaintext Google Titan] - open source secure boot solution
** [https://www.friendlyarm.com/index.php?route=product/product&product_id=284 NanoPi R4S]
* Deprecated projects
* '''Software & firmware'''
** [http://securityrouter.org/wiki/Main_Page SecurityRouter] - OpenBSD distro for network management
** [https://libreboot.org Libreboot] - freedom-respecting boot firmware
<h3>Other OpenBSD-capable boards</h3>
** [http://www.netbsd.org/docs/kernel/porting_netbsd_arm_soc.html Porting NetBSD to ARM926EJ-S] - article on adding support to NetBSD for the same ARM9 core that the Nuvoton NUC980 uses
* [https://www.raspberrypi.org Raspberry Pi]
* '''Other architectures'''
* Pine64 [https://www.pine64.org/pinebook-pro/ PineBookPro] and [https://www.pine64.org/rockpro64/ RockPro64]
** RISC-V:
* [https://www.friendlyarm.com/index.php?route=product/product&product_id=284 NanoPi R4S]
*** OpenBSD on RISC-V: [https://github.com/MengshiLi/openbsd-riscv-notes/blob/master/Porting_OpenBSD_to_RISCV_FinalReport.pdf Porting OpenBSD to RISC-V ISA] (PDF), [https://github.com/MengshiLi/openbsd-riscv-notes GitHub repo]
<h3>Open source / custom silicon</h3>
*** [https://www.sifive.com SiFive] - custom RISC-V manufacturer in the South Bay
* [https://libre-soc.org Libre-SOC] - open source silicon SoC
*** [https://openfive.com/custom-silicon/ OpenFive] - custom RISC-V manufacturer in the South Bay
* [https://www.fossi-foundation.org/projects FOSSi] - Free and Open Source Silicon Foundation
** [https://millcomputing.com Mill Computing] - novel CPU design in the North Bay
* [https://www.skywatertechnology.com Skywater] - low-cost foundry with 180nm and 130nm nodes
* '''Communication & collaboration platforms'''
** [https://github.com/google/skywater-pdk Skywater PDK]
** [https://element.io Element] / [https://matrix.org Matrix] - open source, distributed, secure chat
* [https://efabless.com Efabless] - low-cost custom silicon manufacturing
** [https://github.com/wireapp Wire] - open source, secure chat
* [http://parallel.princeton.edu/openpiton/ Princeton OpenPiton] - open source research microprocessor
** [https://scuttlebutt.nz Scuttlebutt] - open source, distributed, secure social network
* [https://www.youtube.com/watch?v=EczW2IWdnOM Tim 'mithro' Ansell] gives an overview of open source silicon tools
** [https://gitter.im Gitter] - open source chat
<h3>Software & firmware</h3>
* '''Hardware manufacturers'''
* [https://libreboot.org Libreboot] - freedom-respecting boot firmware
** [https://www.sfcircuits.com San Francisco Circuits]
* [http://www.netbsd.org/docs/kernel/porting_netbsd_arm_soc.html Porting NetBSD to ARM926EJ-S] - article on adding support to NetBSD for the same ARM9 core that the Nuvoton NUC980 uses
** [https://www.circuitlaunch.com CircuitLaunch]
<h3>Other architectures</h3>
** [https://oshpark.com OSHPark]
* RISC-V:
** [https://bayareacircuits.com Bay Area Circuits]
** OpenBSD on RISC-V: [https://github.com/MengshiLi/openbsd-riscv-notes/blob/master/Porting_OpenBSD_to_RISCV_FinalReport.pdf Porting OpenBSD to RISC-V ISA] (PDF), [https://github.com/MengshiLi/openbsd-riscv-notes GitHub repo]
** [https://emsolutionstech.com EM Solutions] - PCB assembly
** [https://www.sifive.com SiFive] - custom RISC-V manufacturer in the South Bay
* '''Design tools'''
** [https://openfive.com/custom-silicon/ OpenFive] - custom RISC-V manufacturer in the South Bay
** [https://kicad.org KiCAD] - schematic capture and PCB layout
* [https://millcomputing.com Mill Computing] - novel CPU design in the North Bay
* '''Anti-tampering'''
* [https://www-50.ibm.com/systems/power/openpower/welcome.xhtml POWER9] – open source ISA that's [https://www.openbsd.org/powerpc64.html supported by OpenBSD]
** [https://cercuits.com/transparent-pcb/ Transparent PCB] manufacturer
** [https://github.com/open-power OpenPOWER GitHub repo]
** [https://www.mcmaster.com/tamper-evident-markers Tamper evident markers]
** [https://secure.raptorcs.com/content/BK1SD1/intro.html Raptor Blackbird Secure Desktop] - FOSS server-class POWER9 computer
** Wikipedia article on [https://en.wikipedia.org/wiki/Tamper-evident_technology Tamper Evident Technology]
*** [https://www.osnews.com/story/133093/review-blackbird-secure-desktop-a-fully-open-source-modern-power9-workstation-without-any-proprietary-code/ Review]
** [https://media.ccc.de/v/36c3-10690-open_source_is_insufficient_to_solve_trust_problems_in_hardware#t=2848 "Open Source is insufficient to solve trust problems in hardware"] by Bunnie Huang (video presentation)
<h3>Communication & collaboration platforms</h3>
** [https://dustidentity.com DustIdentity] embedded diamond nanocrystals for system ID
* [https://element.io Element] / [https://matrix.org Matrix] - open source, distributed, secure chat
** [https://spectrum.ieee.org/tech-talk/computing/hardware/chaos-programmable-chips-secure PUFs] (Physically Unclonable Functions) created in silicon on the IC wafer
* [https://github.com/wireapp Wire] - open source, secure chat
*** [https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=9380284 Paper] describing principle
* [https://scuttlebutt.nz Scuttlebutt] - open source, distributed, secure social network
** [https://sharps.org/wp-content/uploads/BECKER-CHES.pdf Stealthy Dopant-Level Hardware Trojans]
* [https://gitter.im Gitter] - open source chat
<h3>Hardware manufacturers</h3>
* [https://www.sfcircuits.com San Francisco Circuits]
* [https://www.circuitlaunch.com CircuitLaunch]
* [https://oshpark.com OSHPark]
* [https://bayareacircuits.com Bay Area Circuits]
* [https://emsolutionstech.com EM Solutions] - PCB assembly
<h3>Design tools</h3>
* [https://kicad.org KiCAD] - schematic capture and PCB layout
<h3>Hardware anti-tampering</h3>
* [https://cercuits.com/transparent-pcb/ Transparent PCB] manufacturer
* [https://www.mcmaster.com/tamper-evident-markers Tamper evident markers]
* Wikipedia article on [https://en.wikipedia.org/wiki/Tamper-evident_technology Tamper Evident Technology]
* [https://media.ccc.de/v/36c3-10690-open_source_is_insufficient_to_solve_trust_problems_in_hardware#t=2848 "Open Source is insufficient to solve trust problems in hardware"] by Bunnie Huang (video presentation)
* [https://dustidentity.com DustIdentity] embedded diamond nanocrystals for system ID
* [https://spectrum.ieee.org/tech-talk/computing/hardware/chaos-programmable-chips-secure PUFs] (Physically Unclonable Functions) created in silicon on the IC wafer
** [https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=9380284 Paper] describing principle
* [https://sharps.org/wp-content/uploads/BECKER-CHES.pdf Stealthy Dopant-Level Hardware Trojans]

Revision as of 15:33, 16 June 2022

Noisebridge | About | Visit | 272 | Manual | Contact | Guilds | Resources | Events | Projects | 5MoF | Meetings | Donate | (Edit)
Guilds | Meta | Code | Electronics | Fabrication | Games | Sewing | Music | AI | Neuro | Philosophy | Funding | Art | Security | Ham | Brew | (Edit)
Security | Bay Area Hackers' Association | OHSNAP | Crypto | SecureDrop | Locksport | Password manager | Aaron Swartz | Security Camera | Edit
OHSNAP before after.png

OHSNAP (Open Hardware for Secure Networks And Privacy), is an open-source platform for building secure networks with a known hardware root of trust.
MAINTAINERS: User:Pion, User:Lizzard. User:James

Open Hardware for Secure Networks And Privacy (OHSNAP)

Before and After OHSNAP Firewall in your network

This is the project page for OHSNAP, an open-source platform for building secure networks with a known hardware root of trust.

Motivation

Virtually all commercially available networking equipment is proprietary and closed-source and cannot be independently verified to be free of malware. There have been documented cases of attackers – sometimes entire nation-states – physically modifying networking equipment and networkable devices in order to exfiltrate data and/or command & control otherwise-trusted systems. This leaves the average individual with little choice but to hope that their home network consists of and is secured by devices which do not phone home or contain other backdoors. Such a situation breaks the guarantee that the user's data and devices remain their sovereign property and instead places control into the hands of manufacturers and governments.

The goal of this project is to produce completely open designs and implementations for critical network infrastructure with auditable roots of trust. By making the hardware design, manufacturing process, and firmware and software stacks fully auditable, it allows users to inspect the entire end-to-end flow of their data and to directly control some or all of the fabrication of the device in order to establish positive provenance.


Device Summary

OHSNAP Firewall platform block diagram
OHSNAP Server platform block diagram

OHSNAP is an exploratory platform for secure hardware. The first two OHSNAP devices will be:

  • OHSNAP Firewall: An embedded computer running an open source firmware and OS. It will expose at least two gigabit Ethernet ports and is intended to be a trusted checkpoint between two untrusted networks. The default software configuration will be OpenBSD running a custom connection monitoring, alerting, and filtering tool.
  • OHSNAP Server: A small, low-power, single-board computer with a single 10/100 Ethernet port. May be used as a server, client, or IoT device. Will run NetBSD.

Design Security Principles

  • No closed-source firmware or software allowed anywhere in the stack
  • Implementation must be independently reproducible by third parties
  • Factory-made PCBs must be avoided if possible or physically produced in the USA when necessary
  • Components should be as supplier-diversified as possible
  • Every phase of manufacturing must be either tamper-proof or tamper-evident
  • Fewer pieces = smaller attack surface

Open Source Repository

All design documentation, schematics, firmware, and software will be available at our GitHub repository.

Meetings

We are currently meeting every Thursday at 16:30 PT (GMT-8) using the Jitsi video platform. If you can't connect: Sometimes our primary Jitsi instance goes down. If you're having connection issues, try our backup Jitsi.

How to Get Involved

The best way to get involved is to come to one of our weekly meetings. All forms of help and areas of expertise are welcome and needed! This is a crossfunctional, multidisciplinary project that requires people with all sorts of talents and interests to collaborate if it is to succeed.

In addition to graciously accepting general help and contributions, we have a current need for specific expertise in the following areas:

  • Low-level software/OS engineering
  • Firmware engineering
  • Hardware engineering
  • Materials science
  • Cryptography
  • Network security
  • Web design

Open Questions

  • How to offer root-of-trust guarantees to non-DIY customers?

Status

OHSNAP Server engineering timeline

June 2022

  • New software initiatives:
    • Core functionality of OHSNAP Firewall will require custom software on the firewall and optionally clients. This will also be a standalone software product that's useful outside of OHSNAP hardware.
      • Firewall software monitors and filters new connections to/from all downstream clients (hooks into OpenBSD's pf)
      • Any new connection attempts (which haven't previously been preconfigured via pf rules) from client devices will generate an alert to one or more registered alerting devices and optionally on clients who want local first-responder rights
      • Users allow or deny the connection (either on a native client or on the alerting device) and optionally save their decision as a rule.
      • All connections can be logged and searched for later forensic discovery or trend monitoring.
    • oort (OHSNAP OpenBSD Rapid Provisioning Tools): A toolkit that can automate the provisioning of multiple devices with OpenBSD. Useful for both speeding up our debug cycles and for final imaging of hardware deliverables. Will also be a standalone software product that's usable outside of OHSNAP hardware.
  • OHSNAP Server (with an appropriate UART LCD, Allow/Deny pushbuttons, and optional buzzer/vibration motor) can be used as a standalone alerting device for above.
  • Created new top-level dev repositories (server, firewall, oort) along with a source control policy and workflow for the team.
  • Separate new repo (docs) for links and other useful documentation of our research

February 2022

  • Continuing discussion on OHSNAP Firewall architecture
    • May consider adopting TFC-like separation between RGMII send & receive signals
      • Requires splitting design into 2-3 independent boards.
      • Could require some form of hardware authentication between boards. Key generated by using at time of firmware compilation & manual flashing.
      • User can optionally be required to add final Ethernet jacks manually, enhancing security
  • Restarted working on OHSNAP Server hardware
    • Continuing PoC board layout
    • Minor schematic updates include updating to KiCAD 6, creating a pin mapping for tracking resource allocation in firmware, and adding an SD card module for bootloading.
    • Following updates of Voltera NOVA (desktop additive flexible electronics manufacturing) and Efabless (open source custom silicon manufacturing) for possible use in project
  • Added a graphic showing OHSNAP Firewall as part of a user's network ecosystem.

November 11, 2021

Had some discussion on the OHSNAP Server boot process

  • HW will have SPI flash, one SD card, and one USB port
  • Primary bootloader will reside in SPI flash (may be updated through JTAG)
  • Boot installer image or installed OS from SD card
  • Installer will write OS to USB flash drive
  • SW Development sequence:
    • 1. Port NetBSD to NUC980
    • 2. Write USB driver layer
    • 3. Write SD card driver layer

Hardware To-Do: Need to add SD card

October 28, 2021

  • Work has started on adding QEMU support for the NUC980 chip which we are using in OHSNAP Server. Development is currently on the develop branch.

August 10, 2021

  • Work on Nuvoton dev board PCB layout is in progress; expected checkpoint in September
  • Identified current main need: We need to grow the team to help solve major root-of-trust questions
    • Need an easily digestible web presence (like a one-pager with an easy URL) to quickly show prospective collaborators
      • We need a web designer to help with this!
    • Move to a more distributed collaboration platform (Element/Matrix)
    • Get more visuals (diagrams, photos) of plan and works-in-progress
    • Continue to reach out to potential contributors and allies
      • Can YOU help?
  • Discussed alternatives to SiFive/OpenFive + related projects


August 3, 2021

Meeting notes with SiFive / OpenFive:

  • Can they do it? Yes: OpenFive has the capabilities to package both custom and standard cores to customer specs and can easily put the UC74-MC into a QFP package.
  • Costs for any ASIC work are divided between IP and manufacturing.
    • Manufacturing cost scales inversely with process node.
      • 28nm process is roughly $1M for masks and another $1M for tapeout + packaging + validation. This is for a typical/minimum order quantity of 10,000.
        • For comparison, 5nm processes are on the order of $20M+.
      • The maximum process node is determined by design requirements (particularly max frequency, die size, power consumption, thermal dissipation). For OHSNAP Firewall our estimated lower frequency is ~700 MHz. This most likely means a maximum of 28nm process.
      • Re-spins are often necessary for most clients due to silicon bugs. For 28nm, this adds a typical $150-200k for a 3 layer respin.
    • IP cost depends on what core we use and what options we select. The stock UC74-MC core that we were considering is about ~$1M. Additional features drive the price up. Switching from -MC (multicore) to single core only saves a modest amount of money.
      • They offered a lower-end alternative to the U7-series: The U5 series. It is also Linux capable but has a shorter pipeline and so isn’t as performant. Typical IP is $500k.
  • Time: Typical time between submitting final RTL designs and start of tapeout is 1 year. This includes 1 week to generate masks, 10-12 weeks to process the silicon (for 28nm), postprocessing time (may not be necessary for QFP packages), die assembly at the package assembly site, and testing. Add another 4-5 months until we can receive engineering samples. Total time for the entire process from start to ramp is 1.5-2 years.
  • Tamper evidence / manufacturing control:
    • OpenFive uses two foundries: TSMC and GlobalFoundries. Our requirement for domestic manufacturing eliminates TSMC and 6 out of the 9 GlobalFoundries sites, but the remaining 3 are viable candidates. According to this article, two of those can hit 28nm and both are in New York.
    • Three main arenas to consider: Silicon manufacturing, packaging + assembly, and testing.
    • OpenFive transports the products themselves.
    • They would be open to us installing an observer to monitor production (for a fee).
    • They would be open to providing us the results of Layout-Versus-Schematic (LVS) tests.
  • Development cycle
    • 1. Pre-silicon: Emulation
      • QEMU
      • RTL
      • FPGA Bitstream
      • System C
    • 2. Dev board
      • For U5/U7 series chips: HiFive Unmatched
      • Do bulk of development while waiting for engineering samples
    • 3. Engineering samples
      • Finish main development
      • Debug debug debug
      • Possibly re-spin Rev B silicon
  • They briefly mentioned SiFive Shield and WorldGuard, two architectural technologies for enhanced silicon security. While these don’t solve our problem of trusted manufacturing, they are definitely useful layers to add to our secure ecosystem.
  • Summary: They seemed ready to engage with us but the cost is the big question. They were impressed by our mission and our passion and wanted to know more about the device, its use cases, and our business model. We told them that they are our Plan A and we really want to work with them but their cost estimates were an order of magnitude greater than we anticipated so we have to go back to the drawing board.

July 27, 2021

Preparing for meeting with SiFive to discuss custom packaging of the UC74-MC RISC-V core.

July 9, 2021

Explored RISC-V option. Many advantages:

  • We can reduce the attack surface by including only the silicon IP blocks that we need
  • Eliminates the requirement for outsourcing PCB manufacture
  • One chip can serve as the base platform for all OHSNAP projects (firewall, server, TFC, etc.) as well as being adoptable by third parties
  • Potentially improves best-case root-of-trust guarantee (but this requires a lot of thought and care)
  • Extends the idea of Open Hardware beyond traditional limits!

Main barriers:

  • High cost of entry
  • Need vendor buy-in for enhanced tamper evident manufacturing practices
  • Significant development effort

We will deliberate and vote on this. If in consensus, next step is to engage vendor(s) and start understanding the height of the above barriers.

July 2, 2021

Examination of the OpenBSD source and mailing list archives shows that support for pre-v7 ARM architectures was intentionally removed from the OS. According to this table, the NUC980's ARM926EJ core is ARMv5-based and therefore unsupported.

Adding support for ARMv5 to OpenBSD is out of scope for this project and we're not aware of any plans by the OpenBSD developers to do so either. This means that we can't use the NUC980 with OpenBSD for the OHSNAP Server and leaves us with these choices:

  1. Pause the OHSNAP Server project and continue to focus on OHSNAP Firewall
  2. Resume searching for OpenBSD-compatible non-BGA chips that are similar to the Nuvoton (unlikely to exist on the market, as we have been searching for a while and chips like this tend to be older designs and therefore unlikely to be supported)
  3. Migrate to a different OS which has ARM926EJ support, such as NetBSD or Linux. Both are considered less secure than OpenBSD, the latter unacceptably so.
  4. With newly added and quickly expanding RISC-V support in OpenBSD, we may want to consider that architecture more seriously. One key advantage of open source silicon is that we can create a custom, high-performance chip and put it in a QFP package with only the pins exposed that we would need. Next steps would be getting ballpark estimates of taping out our own silicon from local vendors.

June 25, 2021

  • Created v0.1 schematic for NUC980-based dev board. Will first be used with bare metal firmware.
  • Ordered samples of NUC980DR61Y and the Nuvoton Chili dev board. The latter will be used to test firmware and Linux and to begin the OpenBSD port.
  • Found article on porting NetBSD to ARM926EJ-S, the same core as in the Nuvoton chip.

June 18, 2021

We've settled on a roadmap:

  1. Create a fully homebuilt, low-performance 10/100 single Ethernet board that is capable of running OpenBSD ("Option A" from March 27, 2021). This approach will:
    1. Provide us with hands-on experience in designing, building, debugging our own hardware.
    2. Give us a testbed for experimenting with secure manufacturing practices.
    3. Require us to learn how to port OpenBSD to a new chip and board.
    4. Provide a ready-to-use reference design for other, related secure manufacturing projects at Noisebridge and beyond.
    5. Produce a physical deliverable that will create buy-in and grow our ecosystem.
  2. In parallel, we will continue to identify and engage potential vendors for high-performance designs which require securely outsourcing manufacturing.

The first low-performance design will be built around the Nuvoton NUC980DR61Y (see March 20, 2021). This will come in stages:

  1. HWPOC (Hardware Proof Of Concept): A minimal board designed to allow creating bare metal firmware to test basic design and in-house manufacturing on the Voltera PCB printer. We may attempt to boot Linux from this board, as there is kernel support for it and the design will be derived from the Nuvoton Chili dev board.
  2. SWPOC (Software Proof of Concept): Use the Nuvoton Chili board to port OpenBSD to the target chip.
  3. Proto: Boot OpenBSD from the minimal in-house board.
  4. EVT: First in-house board featuring Ethernet and all planned features. Still contains debugging connections.
  5. DVT: Finalized PCB. Begin testing low-qty mass production using pick & place machine.
  6. PVT: Final production units. Goes on to ramp build.

May 8, 2021

  • Continued exploration of threat vectors and possible mitigations.
  • More links added to resource list.
  • Setting up Tinfoil Chat as a realtime collaboration platform and exploration of secure methodologies.
  • Setting up OpenBSD on various SBCs and virtual machines to evaluate its fitness for use.

April 3, 2021

Identified three regimes for threat models, some concrete examples, and some hypothetical mitigations:

Threat Regime Example Scenario Possible Mitigations
During assembly Manufacturer tampering
  • PCB x-ray inspection
  • Translucent PCB substrate for visual inspection
  • Encase PCB in glitter epoxy
  • Tamper-evident paint
  • Apply heat-sensitive paint to PCB (detects soldering)
  • Vacuum-sealed enclosure with onetime-use pressure sensor
  • Serial numbers in ROM on ICs
  • Use common enough chip to reduce chance of silicon tampering
  • Pogo board testing
During transit to end-user Vendor/reseller tampering
Mail interdiction
Embed device in another device
At customer's install site Evil user tampering Live system intrusion detection (examples??)
Routine pogo board testing
Passive monitoring, e.g.,
  • audio/video monitoring of keypresses
  • RF leakage analysis
  • power analysis
???

March 27, 2021

Still looking for a non-BGA ARM processor that contains two Ethernet PHYs and has 0.5mm or greater lead pitch (for Voltera V-1 PCB production. We are now considering three options for design:

  • (A) Non-BGA ARM processor on a homemade mainboard [simplest but limited to non-gigabit speeds; may require SW porting]
  • (B) BGA ARM processor on a fabbed carrier board attached to a homemade mainboard [more complicated but higher performance and easier to source; may avoid SW porting]
    • E.g., NXP i.MX6SoloX
    • Unpopulated carrier board will need to be examined, probably via x-ray, after production
    • Carrier board reflow is still done by us
    • Still need to look for more ways to increase provenance
  • (C) Use an existing open source BGA-based project (e.g., Kosagi Novena) and modify it for trusted manufacturing [complexity likely between (A) and (B)]
    • Will still likely need a BGA carrier board from (B) but may reduce R&D costs of the project

Infrastructure

We’ll be setting up an IRC server in parallel to testing local deployment of the Element secure chat system.

March 20, 2021

In order to make forward progress on the hardware, we will choose OpenBSD as the initial operating system and an ARM-based architecture for the CPU. Users will be able to use their own ARM-compatible OS if they choose, including Linux and Plan9 (if software support is added).

We will first create a Software-Intent Proof of Concept (SIPoC), which will be an OpenBSD-based router/firewall running on a commercially available SBC. We may then want, as a Proof of Concept (PoC), to port the software stack to the Common Networks nodes at Noisebridge and begin deploying them across sites.

The first version of our custom hardware should be amendable to DIY manufacturing. This means no BGA parts. This severely restricts the list of CPUs/SoCs to older and lower-performance chips, limiting our capability to 100MBps initially. We'll expand this list as we discover more:

List of non-BGA ARM SoCs

March 13, 2021

  • Looking for secure communications platform for project collaboration. Element / Matrix look promising.
  • Possibly partner with CircuitLaunch for local hardware builds?

March 6, 2021

  • Initial meeting. Discussed range of HW/SW design choices.
  • Tentative Project Stages
    • SIPoC: OpenBSD router/firewall on commercial SBC
    • PoC: SW stack on Common Networks
    • Proto 1 build: Low-speed (10/100 Mbps) DIY version
    • Full build: 1 Gbps
  • Explored possible design choices for future versions
    • CPU
      • ARM/ARM64 SoC
      • RISC-V SoC
      • FPGA
      • Specifically no Intel/compatible architectures due to poor security record
    • OS / Application Code
      • OpenBSD
      • Qubes
      • Alpine Linux
      • Plan9
      • Custom FPGA code
    • Trusted manufacturers

Links to Resources

OHSNAP Internal Documentation Repository

Most new links will be added here: docs

Related projects

Other OpenBSD-capable boards

Open source / custom silicon

Software & firmware

Other architectures

Communication & collaboration platforms

  • Element / Matrix - open source, distributed, secure chat
  • Wire - open source, secure chat
  • Scuttlebutt - open source, distributed, secure social network
  • Gitter - open source chat

Hardware manufacturers

Design tools

  • KiCAD - schematic capture and PCB layout

Hardware anti-tampering