The Robot World Cup Initiative (RoboCup) is an international joint project to promote AI, robotics, and related field. It provides a standard platform for robotic soccer game which includes a vision system, a strategic play algorithm and small mobile robots. The aim of the vision system for RoboCup small robot league is to track and predict the motion states of 6 agents and a ball, and send the states information to the Artificial Intelligence module. This paper presents a design and implementation of a real-time, robust global vision system. The two main efforts are realized in this study, design and implementation of multi robot structure and the construction and tasting of the tracking and path planning algorithm. In the tracking phase, the color-based segmentation is used to locate the interesting objects in the image; the blob analysis is adopted to calculate the positions of the objects in the image and the noise is filtered out; and the linear filter is adopted to track and predict information of the states. It has been planned that the information gathered using this study can be used in multiple agent robotics applications. © Springer-Verlag Berlin Heidelberg 2006.