From Sudek's favourite Castle and Royal Garden the Ball Games Hall was damaged the most (sometimes called the Great Ball Games Hall). The building, originally intended for the playing of ball games, is a Renaissance site from 1567–1569 designed by the court architect and builder Bonifác Wolmuth. It was part of a set of buildings in the Royal Garden, such as Queen Anna's Summer Palace, an orangery and greenhouses built in the 16th century.
The Ball Games Hall burned down at the end of the May Uprising, on 9 May 1945. It was supposedly set on fire by occupiers escaping Prague. It had been damaged almost two centuries before during Prussian bombardments and as a consequence caught on fire. The devastating fire in 1945 swept through the roof and beams fell into the interior. The Ball Games Hall was restored in 1948–1950 by Pavel Janák with the participation of Bedřich Hacar. The restoration works and completion of the sgraffitos on the exterior were carried out by the sculptor Josef Wagner, a close friend of Sudek's.In addition to typical motifs, there is also one curiosity on the facade – a socialist allegory of industry with the emblem of the five-year plan as a new addition from the period of this reconstruction.
Nowadays, the Ball Games Hall is the pride of the Royal Garden, its present appearance is the result of the latest restoration works from the 1990s.