Detecting damaged car parts

Question

I am trying to build a system that on providing an image of a car can assess the damage percentage of it and also find out which parts are damaged in the car.

Is there any possible way to do this using Python and open-cv or tensorflow ?

The GitHub repositories I found that were relevant to my work are these

https://github.com/VakhoQ/damage-car-detector/tree/master/DamageCarDetector

https://github.com/neokt/car-damage-detective

But what they provide is a qualitative output( like they say the car damage is high or low), I wanted to print out a quantitative output( percentage of damage ) along with the individual part names which are damaged

Is this possible ?

If so please help me out.

Thank you.

Answer 1

To extend the good answers given by @yves-daoust: It is not a trivial task and you should not try to do it at once with one single approach.

You should question yourself how a human with a comparable task, ie say an expert who reviews these cars after a leasing contract, proceeds with this. Then you have to formulate requirements and also restrictions for your system.

For instance, an expert first checks for any visual occurences and rates these, then they may check technical issues which may well be hidden from optical sensors (ie if the car is drivable, driving a round and estimate if the engine is running smoothly, the steering geometry is aligned (ie if the car manages to stay in line), if there are any minor vibrations which should not be there and so on) and they may also apply force (trying to manually shake the wheels to check if the bearings are ok).

If you define your measurement system as restricted to just a normal camera sensor, you are somewhat limited within to what extend your system is able to deliver.

If you just want to spot cosmetic damages, ie classification of scratches in paint and rims, I'd say a state of the art machine vision application should be able to help you to some extent:

• First you'd need to detect the scratches. Bear in mind that visibility of scratches, especially in the field with changing conditions (sunlight) may be a very hard to impossible task for a cheap sensor. Ie to cope with reflections a system might need to make use of polarizing filters, special effect paints may interfere with your optical system in a way you are not able to spot anything.

• Secondly, after you detect the position and dimension of these scratches in the camera coordinates, you need to transform them into real world coordinates for getting to know the real dimensions of these scratches. It would also be of great use to know the exact location of the scratch on the car (which would require a digital twin of the car - which is not to be trivially done anymore).

• After determining the extent of the scratch and its position on the car, you need to apply a cost model. Because some car parts are easily fixable, say a scratch in the bumper, just respray the bumper, but scratch in the C-Pillar easily is a repaint for the whole back quarter if it should not be noticeable anymore.

Same goes with bigger scratches / cracks: The optical detection model needs to be able to distinguish between scratches and cracks (which is very hard to do, just by looking at it) and then the cost model can infer the cost ie if a bumper needs just respray or needs complete replacement (because it is cracked and not just scratched). This cost model may seem to be easy but bear in mind this needs to be adopted to every car you "scan". Because one cheap damage for the one car body might be a very hard to fix damage for a different car body. I'd say this might even be harder than to spot the inital scratches because you'd need to obtain the construction plans/repair part lists (the repair handbooks / repair part lists are mostly accessible if you are a registered mechanic but they might cost licensing fees) of any vehicle you want to quote.

You see, this is a very complex problem which is composed of multiple hard sub-problems. The easiest or probably the best way to do this would be to do a bottom up approach, ie starting with a simple "scratch detector" which just spots scratches in paint. Then go from there and you easily see what is possible and what is not