There is actually a way to nullify an agent, if the opponent places its own agent on one of its tiles that you have your agent at, yours will be nullified. This is called counter-espionage, where you can actually use your agents to stop other agents working in your own buildings. So to anwsering your question, it could be that the AI is producing agents at the same rate as yours and its using it to detect your agents. On a side note, I woudn't use agents in the begining and rush those agents simply because their cheap and fast to train. I would use them either at the mid or end-game where you have a considerable amount and can use them effectively.