The Issues of False Alarm Rate and Hit Rate
While many porn-detection software claimed about nearly 100% hit rate, the writers of these software intentionally or unintentionally hide the other side of the story; namely, such a high hit rate is only achieved by increasing dramatically the false-alarm rate. In a typical scenario of skin-tone based porn-detection software, a 50% false-alarm rate is very typical for a unbiased image database reflecting the activities of human society such as basketball game and busy airport. Before we go into the details, let us set a standard test for those porn detecting software. At Yang’s, we constructed a bench mark image test set for all porn detection software and also for our own engineers to test the efficiency of their software. This benchmark test image set was constructed based on many facts of the real Internet image base. The following facts are taken into account:
1. People have different skin tones.
2. A DIY porn picture is usually has uncontrolled color tones and saturation.
3. It is difficult to tell a porn from a clear image where a woman wear only bra and underwear.
4. Black and white porn images do exist.
5. No one looking into sky or an open sea to find porn pictures.
PornSeer Pro uses two parameters “Threshold” and “Decision” to tune the relation between hit rate and false alarm rate.
The testing results against the benchmark image database that will be addressed late in this note show that PornSeer Pro are 25.5% better than the mainstream skin-tone based porn-detection software.
We then discuss the relation between the two parameters and the hit rate and false alarm rate.
Threshold |
Threshold defines the sensitivity of detecting a pornographic feature such as a breast or a pubic hair region. The smaller the threshold is chosen, the more sensitive PornSeer detects a porn feature. However, we can not reduce the threshold too far below 200 because when the threshold is too small, many unrelated features can mix up with those useful features. For example, a breast region is typically has a lighter and smooth surrounding region while at the center a dark and rough region. PornSeer Pro takes advantage of this spatial configuration to distinguish a breast from a dish or from a light bulb. However, if the threshold is too small, the boundary between a breast and a dish will become much blur than when the threshold is bigger. If PornSeer needs to put a great effort onto the central regions of a breast for the purpose of distinguishing a breast from a dish, it needs a relative big threshold to do so. On the other hand, the threshold can not be too big. Otherwise, a breast region with a lighter nipple will be classified as a dish. Our experience is that the optimal value for the threshold lies anywhere between 200 to 500 based on the expect false alarm rate from the range of 50% to 30%. |
Decision |
PornSeer Pro was designed to decouple the connection between the parameters of Threshold and Decision as much as possible. However, the relation between these two parameters does exist. We observed that a small Threshold is usually used together with a small Decision to keep the same false alarm rate and the hit rate. Yet, since the connection is no very strong, the user can tune them separately. A big Decision can reduce the false alarm rate and yet, reduces the sensitivity as well. If in the video we can find lots of faces showing up or someone wearing a bra moving around, a 600 level Decision will be most likely to trigger a signal of porn. While a 1000 level Decision will be much less likely to react upon face region, it might fail to detect pornographic scenes without breasts or vulva region. Therefore, it will depend on the nature of the video clip and the priority of the detecting task to guide the choice of Decision. If one wants to have a high hit rate, he might want to choose a low Decision level around 500. If one wants to get a very low false alarm rate, he might want to increase the Decision level to 1000 or more. |
Benchmark Image Database
The benchmark image database consists of 400 images; 200 porno images selected from over 100,000 porn images and 200 clear images selected from over 1,000,000 images. The following are 200 clear images
 |
 |
 |
 |
 |
 |
| The following are 200 porn images in the benchmark image data base. |
 |
 |
 |
 |
 |
 |
| As a typical skin-tone based porn-detection software, the software PixAlert Auditor 3.1 found 90 “porn images” from the clear image set listed as follows. |
 |
 |
 |
| While PixAlert Auditor 3.1 has almost 50% false alarm rate, one should expect it has a high hit rate at the same time, however, it turns out that this software only found 109 porn images from the porn image set listed as follows. |
 |
 |
 |
 |
| When we set the PornSeer Pro to working in similar condition; namely, when the parameters of PornSeer Pro were set to find 110 porn pictures from the porn image base, PornSeer Pro only has 67 false alarms, which is 23 less than what PixAlert Auditor 3.1 did. This is a 25.5% improvement of the detection results achieved by PornSeer Pro. |
The academic users can ask for a low-priced the academic developers’ kit and free source codes in C\C++ for
non-commercial purpose. For commercial applications, the license
fee depends on the functions needed. For more information on this technology,
please send your request to
