检测图像中的手绘电路组件，检测文本，建立连接树

Question

I want to know, how I should process hand drawn images of circuit diagrams, for the sake of digitizing the drawn circuit and eventually simulating it.

The input to my program would be a regular picture (smartphone etc..) and the finished output should be a simulation of all possible values in the circuit (not covered / required here)

Basically all I would need to be able to detect are electrical components with a fixed number of connections (2 connections eg R,L,C,Diode) and the lines connecting them.

I already have a pretrained neural network for detecting which type of component it is. The part where I struggle is, how do I get bounding boxes around the components so I can classify them with my NN? I tried several approaches using Contouring and Object detection using OpenCV (eg. FindContours , ConnectedComponentsWithStats ) but I cant seem to get it to detect only the components, and not the text or connecting lines between components.

Basically what I want is the following:

Given this Input Image (not hand-drawn for sake of readability)

I would like to know:

• How many components are there?
• Where are the bounding boxes of the components?

Basically These bounding boxes

This is used to extract components and classify them with the model I already have.

Furthermore, I would like to extract the Text closest to any component, so that I can read the values of each component. I have already managed to do OCR with the help of tesseract-ocr , so if I can get bounding boxes around the text I can easily read the values.

Like this

But the part I struggle with the most, is finding out which component is connected to which other component, I am unsure how I should approach this. Its really hard to find something googling my problem, and not certain how I should describe this problem in general. But overall, I need enough information to be able to simulate the circuit with the Matrix-Simulations (basic DC-Analysis).

I do not explicitly require code, I need general guidance to solving this problem. Or maybe even links to research papers attacking similar problems.

Answer 1

"Every problem is just a good dataset away from being demolished" source.

There are several electronic symbols that you will need to detect. A modern approach to classifying the symbols would be to use a neural network. To train this model, you will need to create a dataset of hand-drawn electronic symbols. Classifying the electronic symbols will be similar to handwritten digit classification .

Before the symbols can be classified by a neural network model, the image will have to be segmented. Individual components (diode, capacitor, resistor, etc...) will need to be identified, and labeled with bounding boxes.

The complexity of this task depends on the quality of your source images. Images that are created using a scanner (instead of a camera) will be much easier to work with.

This task can be accomplished with OpenCV and Python. This is a breakdown of the sub-tasks:

• Mobile Document Scanning
• Component segmentation
• Component classification
• OCR