USB C industrial camera with Interchangeable C mount Lens, Interchangeable MIPI Sensor with Lattice Crosslink NX FPGA Cypress FX3 USB 3.0 controller

FPGA ISP Pipeline Specifications

MPI CSI receiver ISP has No virtual restriction on supported frame rate or resolution. Tested more than 4K with IMX477 4056x3040. Can do 8K with around 30FPS or even higher than that as long as FPGA is fast enough for needed frame rate and FPGA/Board has enough memory to be able to store minimum 4 Line worth of pixels. Output Clock is independent of MIPI clock. Easily Portable code to Xilinx or any other FPGA, No Vendor specific components has been used except for the PHY itself which can be replaced by other vendor's DDR phy and Embedded Block RAM. Only Debayer/Demosaic and Output reformatter need Block RAM. Block ram can also be replaced vendor's RAM. Auto detection of RAW pixel width supporting different camera sensors and sensor modes without FPGA reconfiguration.

Speed

• Supports MIPI bus clock 900Mbitsps Per lane with upto 4 Lanes, Total 3.6Gbitsps Sensor bit stream, Has been Tested upto 900Mbitsps with 8x Gear.
  
• Pixel Processing pipeline with 2,4 or 8 Pixel per clock can reach more than 110Mhz with Lattice Crosslink-NX LIFCL-40 High Speed, So basically Can process upto 880 MegaPixels per second. With this can reach Around 120FPS with 4K resolution and around 30 FPS with 8K. Or even 3000 FPS with 640 x 480 as long as Camera and MIPI Wire allows. With Different Faster FPGA speed will be more.
  
• FPGA Oputput Pipeline is decoupled from MIPI clock, runs on output clock, It feeds into Cypress FX3 32bit GPIF can do Max 160Mhz. Cyress FX3's specs limits max GPIF clock to 100Mhz.
  

Configurability

• Selectable max RAW pixel width
  FPGA Design is configurable with parameters to support pixel depth from RAW10 to RAW14 or Virtually any bit depth even 16bit RAW when it becomes a MIPI Specs. Parameter specify maximum pixel width that is supported while module auto detect package type at runtime with RAW14 selected as max pixel width, RAW10, RAW12 and RAW14 will be automatically detected and processed
  
• Selectable number of MIPI lanes
  With just definition of Parameter value number of lane is also configurable between 2 or 4 MIPI lanes.
  
• Selectable Pipeline Size
  Pipeline is Configurable with a parameter to Process 2,4 or 8 Pixel. 2 Pixel Per Clock is only available with 2 Lane MIPI, while 8 Pixel Per Clock is only available with 4 Lanes.
  
• Selectable MIPI Gear Ratio
  User can select weather to operate MIPI/DDR phy in 16x or 8x Gear ratio. Most DDR/MIPI phy supports 8x Gear while few do support 16x gear.
  Block RAM and DDR PHY IPs need to be manually regenerated if Gear, pixel width , lane or PPC is changed.
  
• Selectable MIPI continuous clock mode
  User and select between MIPI clock lp based Frame sync or Frame start and frame stop packt based frame sync. Some MIPI cameras do not support going to LP mode while frame blank occur, With this option user can enable Frame Start and Frame stop detection, to have a frame sync.
• Selectable ROM based Sample Generator
  For ISP debuging ROM based sample generator can be activated. Two ROM lines are there have both even and odd line to full image test.

Tests

IMX477

• 4 Lane 12 bit IMX477
  4056x3040 15 FPS Full Sensor
  2028x1520 60 FPS Full Sensor Binned 2x2
  2028x1080 100 FPS
  

• 4 Lane 10 bit IMX477
  1332x990 150 FPS Binned 4x4
  640x480 400 FPS Binned 4x4
  640x128 2400 FPS Binned 4x4
  

• 2 Lane 12 bit IMX477
  4056x3040 10 FPS Full Sensor
  2028x1520 35 FPS Full Sensor Binned 2x2
  2028x1080 50 FPS
  

• 2 Lane 10 bit IMX477
  1332x990 100 FPS Binned 4x4
  640x480 200 FPS Binned 4x4
  640x128 1200 FPS Binned 4x4
  

IMX219

• 4 Lane 10 bit IMX219
  3280x2464 30 FPS
  1280x720 240 FPS
  1920x1080 120 FPS
  640x480 400 FPS
  640x128 1300 FPS
  640x80 2000 FPS
  

• 2 Lane 10 bit IMX219
  3280x2464 15 FPS
  1280x720 60 FPS
  1280x720 120 FPS
  1920x1080 60 FPS
  640x480 30 FPS
  640x480 200 FPS
  640x128 600 FPS
  640x80 900 FPS
  

IMX290/IMX327/IMX462

• 4 Lane 12 bit
  1280x720 120 FPS
  1920x1080 120 FPS
  

• 2 Lane 12 bit
  1280x720 60 FPS
  1920x1080 60 FPS
  

Project Blog post

https://www.circuitvalley.com/2022/06/pensource-usb-c-industrial-camera-c-mount-fpga-imx-mipi-usb-3-crosslinknx.html

Shield:

This work is licensed under a Creative Commons Attribution 4.0 International License.