Working Principles

A computational noun usually consists of two parts, a BEING and its attribute values. For example, "big apple" is a computational noun with a "BEING = apple" and the attribute value of the size of the BEING to be "big". Today it is widely known that the attribute value such as "big" can be easily implemented into computers by using fuzzy sets which were invented by a UCB EECS professor, Professor Zadeh, in 1965. Professor Zadeh called the BEING as linguistics variables and put the attribute values as the values that we can normally assign to a variable. Since we are now not treat nouns as isolated component of a natural language, we prefer to treat the BEING and its attribute value as a whole, and called it the computational nouns.

On the other hand, a computational verb consists of two parts too, a BECOMING and its modifiers. For example, "smile happily" is a computational verb where "BECOMING = smile" and the modifier is "happily". Today, we know that computational verbs can be implemented into computers by using computational verbs which were invented by a UCB EECS researcher, Mr. Yang, in 1997. Mr. Yang is now the chief-scientist of Yang’s Scientific Research Institute and continue to develop computational verb theory into a much wider framework.

Computational verbs and computational nouns are two indispensable components of the first measurable linguistics called physical linguistics. In physical linguistics, each word can be measured and computerized. The property of physical linguistics is extremely useful to build a search engine connected direct to signals, rather than texts. For example, trying to search a nose from a facial image, one must now the measuring benchmarks of the nose itself, the measuring relation between a nose and two eyes and the mouth. Without the measuring benchmarks and the relations between those BEINGs, there is no way to make sure that we can find the nose and make a search index for it.

Therefore, to index an image, we need a few computational nouns to describe the measuring benchmarks and a few computational verbs to model the measuring relations. The isolated and segmented mathematical models of computational nouns and computational verbs don’t have any advantage over other conventional equivalents. Let us put is in this way, if you learn a foreign language and only know how to speak “hello” and “goodbye”, then you can not use that foreign language to do any thinking. However, think about your native tongue, you have an entire network of words and each of then either connected to something or some action, you can connect all those things via all those actions painlessly. Therefore, united, all computational nouns and computational verbs gain power. How to unite all computational nouns and computational verbs? It is simple, we use a semantic network called physical linguistic network.

What is a physical linguistics network? It is a highly organized structure to connect many signals in the physical world via different kinds of measuring mechanisms with symbols. The computational cognition emerges when the measurements can be directly injected into and mix-up with texts. Therefore are two consequences when we use a physical linguistics. First, the texts that are processed by the computers become very context-rich and reflect the basic aspects of cognition; namely, emotional and personal. Second, the physical measurements will talk to logic flow directly and therefore, there will be no crisp right or wrong, up or down and black and white because everything must be related to the sensor outputs or the signal.

With a physical linguistic network, all features in an image can be organized in a very concise and highly-efficient way just like the image signals represented in human brains. For example, each components of a human face can be measured into a few computational nouns and related into a few computational verbs. When we search a lip, instead of recognizing the lip again in the image, we just search something on a face, below nose and usually appear as dark region, etc. Yes, you are right, we are searching the story of the “image-to-story” results. Before we can tell you the reason why physical linguistics is very suitable for image indexing and searching, let us first review the difficulties of image searches.

Why searching images is that kind of difficult? There are two reasons:

1. Our computers are not fast enough. If we can get lots of massive parallel computers as powerful as human brains on everyone's desk, the most difficult image search problem can be solved with even the worst algorithm; namely, match every pair of images you have :-) Of course, with the dramatic progresses made in computer industry recently, we have very few solid arguments towards the speed side, then,

2. Our image understanding algorithms are not smart enough. It is difficult to believe and yet it is the fact that after human beings put lots of man-power-years into these algorithms, it seems to be hopeless comparing with human brains. We have nothing to say if someone blames us on this.

With a fast enough computer and big enough image base at hand, what should we do to develop a smarter image search engine? There are two directions we shall look at:

1. Take the full advantages of the fast CPU's and the evidences found in human visual systems. First, a human brain never let the raw data to mess up its network. This shows us a way to process image; namely, we need to get rid of the raw pixels as early as possible along the processing route.

2. Human eye take a constant flow of visual signals and keep a very flexible framework to learn and evolve the visual experiences along time. We need to build an image understanding engine that have a life and can learn and evolve when its image base growing bigger.

Sounds like a scientific fiction, right? How can we do it in that way? PicSeer is the first image understanding framework to attack this problem in this way by using the following strategies:

1. Represent an image by using natural language with the help of the latest, advanced research results from physical linguistics. By doing this, the raw pixels of each image need only to process once and then all the other image understanding tasks are finished within the framework of physical linguistics. This is very much like a human being to describe an image using natural language and save lots of bytes and CPU time.

2. Using a semantic network of computational nouns and computational verbs to represented each image operations in the physical linguistics. By doing so, the learning results are encoded in natural language and can be entirely understood by human experts. The co-evolution of computer and human experts can be implemented easily in the language level, rather than in the signal level. Since in the core of fuzzy theory and computational verb theory, tons of tedious mathematics reside, we normal people don't care about much mathematics, therefore, let us use examples to show the PicSeer way and leave mathematics to the academic world.

The first ALPHA version of the PicSeer Demo was released on Jan.31, 2006 and it only released its "human female" database to the public at that time. On Feb. 11, 2006, the panda database released to public. The search experiences were much different to other image search engines. The following are examples of how to do the searches.

PicSeer Demo works like a standard web browser. However, there is something different. At the normal place of the URL address bar, a "Freeform Search" bar is implemented. You can type the search text for your image onto the search bar and PicSeer will take care of the decomposition of your freeform text into what it will work with. For example, if you want to find images of woman with big face and the face should be centered on the images, then you can type "woman big center " onto the Freeform Search bar as follow.



Then hit "Enter" key, you will get



Wait a minute, what the heck does "CN0=[human female] MO0=[+, big, central,] for search string [woman big center]" mean? The bad news is that you don't understand it. The good news is that you don't need to understand it. It is before long or it will take a while (depends on how fast your CPU is) the following search results will be shown up:



Please note that the PicSeer ALPHA-1.0.0.1 version only has a human female database companying it. The images base is generated from some 900,000 to 1,000,000 images grabbed from the web. The PicSeer is evolving fast and it is worth to bookmark this page and check the new results more often.

Scheduled and Ongoing Tasks

Applications