Abhijeet Sinha

Projects

Brain-Inspired Attention for Object Counting

A model that counts objects the way people do. It looks around in quick jumps, keeps a running tally, and stops once it is sure.

In one sentence: instead of taking in the whole image at once, the model takes a series of small glimpses, learns where to look next by trial and error (reinforcement learning), and reaches 92.1% counting accuracy.

The idea

When you count the apples in a bowl, you don’t take in the whole bowl at once. Your eyes jump from one spot to the next (these jumps are called saccades), you keep a mental tally, and you stop once you’re confident. This project builds a neural network that counts in the same way.

Figure 1 · Counting by looking around

1234a sequence of glimpsesglimpse(small window)recurrent memoryof what was seenSaccade network:where to look nextClassification net:how many so far?confident enough? yes → stop and answerno → look again
Instead of processing the whole image at once, the model sees only a small window at a time. It decides where to jump next, keeps a running memory, and stops as soon as it is confident about the count.

How it works

The model is built around a recurrent network that remembers what it has seen so far, plus two heads:

  • Classification network: after each glimpse it estimates how many target objects are in the image.
  • Saccade network: it decides where the next glimpse should land.

When the model’s confidence in its best guess passes a threshold, it stops looking and answers. Nobody tells the model where it should look. It gets a positive reward when its final count is correct, and learns its looking strategy from that reward using Q-learning, a standard reinforcement-learning method.

What we found

  • The model counts with 92.1% accuracy.
  • It learns to adapt how long it looks. Images with more objects need more glimpses before the model is confident, which mirrors how people take longer to count larger groups.

Figure 2 · More objects, more looks (schematic)

123456789number of objects in the imageglimpses needed
A sketch of the relationship reported in the paper: to count accurately, the model takes more glimpses when there are more objects, much as people do. The values here are illustrative, not measured.

Why it’s interesting

Looking only where it matters makes the model’s process interpretable, because you can watch its sequence of glimpses, and efficient, because most of the image is never processed in detail. The project linked ideas from neuroscience (attention, saccades, working memory) with reinforcement learning, and was carried out at the Computational Neuroscience Lab, IIT Madras.