He said more than 60 former employees were now working against Ukraine in Russian-occupied areas.
A total of 651 collaboration and treason cases had been opened against law enforcement officials, he added.
The officials, Ivan Bakanov and Iryna Venediktova, have not commented.
In a video address late on Sunday, Mr Zelensky said: "Such an array of crimes against the foundations of the national security of the state... pose very serious questions to the relevant heads [of the two organisations]."
"Each of these questions will receive a proper answer," the Ukrainian president added.
In a later appearance on Ukrainian TV, Andriy Smirnov, one of Mr Zelensky's top advisers, clarified that the officials have not been fired, as initially suggested, but merely removed pending an investigation.
"In order to prevent the potential influence of these two officials on criminal proceedings, which are being investigated in relation to SBU employees who are suspected of cooperation with the aggressor country, and the same applies to the prosecutor's office, the president made appropriate decisions," Mr Smirnov said.
The removal of SBU chief Ivan Bakanov, a childhood friend of Mr Zelensky, follows the high-profile arrest of a former SBU regional head in Crimea, annexed by Russia in 2014. Oleh Kulinych is suspected of treason.
"Everyone who together with him was part of a criminal group that worked in the interests of the Russian Federation will also be held accountable," Mr Zelensky said. "It is about the transfer of secret information to the enemy and other facts of co-operation with the Russian special services."
Senior intelligence officials based in Kherson have also been charged and Mr Zelensky suggested that further action would be taken against other SBU officers.
It has long been assumed Ukraine had a problem with Russian infiltration of its security services and in his speech Mr Zelensky sought to lay out the impact of repeated security breaches.