Broken glass panels were scattered on the streets within a radius of 100 meters from the prime minister's office building. Smoke columns were rising from the buildings after the bomb attack.
Earlier reports said the man, who was described as wearing police uniforms and carrying police identification, started shooting a few minutes after he landed on the island of Utoeya, about 40 kilometers west of Oslo.
The latest report by the Norwegian news agency NTB said that he was wearing a police shirt with an enforcement notice, but not a full police uniform.
Norway's anti-terror special police unit PST joined the Oslo police in investigating both incidents. The police has advised residents of Oslo to stay home.
"It is necessary to avoid big gatherings, to go back home," said a local police officer.
Stoltenberg said that the prime ministers of the other four Nordic countries and Britain have expressed support and solidarity with Norway.
The bomb attack and shooting had aroused concerns in the European Union (EU) and NATO.
President of the European Council Herman Van Rompuy condemned in the strongest terms the bomb blast.
"I am deeply shocked by the bomb blasts this afternoon in Oslo," said Van Rompuy who had sent a message of condolences and solidarity from the European Union to Norwegian Prime Minister and the people of Norway.
EU foreign policy chief Catherine Ashton also "strongly condemned the abhorrent bomb attack in the center of Oslo."
In a message to the Norwegian prime minister, European Commission Jose Manuel Barroso expressed "deepest condolences" to the victims of this despicable attack and their families.
The NATO chief on Friday denounced the bomb explosion and shooting attacks against the alliance member's government and ruling party in Norway.
"On behalf of NATO, I condemn in the strongest possible terms the heinous acts of violence in Norway," Secretary General Anders Fogh Rasmussen said.