In the 1820's the need for a new gaol had arrived. Francis Greenway was commissioned to design a gaol that would overlook Sydney as a constant reminder that Sydney was a convict town. The walls of the jail were built by convicts from 1822-1824. You can still see convict marks on them. The marks were made to show how much work each convict did. Greenway's plans were used for the walls. But because he was an ex-convict, he was taken off the job and his plans for the buildings were not used. Instead, the jail was built using the plans of a jail in Philadelphia. The Plan of the Gaol was like the spokes of a wheel, with wings radiating from a central point leaving narrow segments of space between. Work began on the jail in 1835 and it took 50 years to finish. By 1840 the Governor's residence, one men's cell block and the women's cell block were ready. To move in, the prisoners walked through the streets of Sydney from their old jail at Circular Quay to the new gaol. The first governor was the amazing Mr Henry Keck.

Darlinghurst Gaol then remained the main Sydney penitentiary up until 1914, when it's unwilling inmates were transferred to the new "model prison" at Long Bay. During its long life, Darlinghurst Gaol hosted public executions on a makeshift gallows outside the main gate in Forbes Street, as well as regular "private" executions on the permanent gallows just inside the main walls near the intersection of Darlinghurst Road and Burton Street. Over 70 people were executed here including the bushranger "Captain Moonlight" (aka Andrew George Scott), Jimmy Governor (known as Jimmy Blacksmith in more recent times), and the last woman to hang in NSW, Louisa Collins.

Henry Lawson also did time at Darlinghurst, for drunkenness and non payment of alimony, and recorded his experience in the haunting poem "One Hundred and Three" - his prison number- which was published in 1908. Here he refers to the prison as "Starvinghurst Gaol" because of the meagre rations given to the inmates.

Haphazard development at the Gaol, together with the gross overcrowding above the design capacity of buildings, led to the continuous problems with drainage, security and disease. The necessity of walled courtyards (to segregate prisoners and to prevent mass rushes at the gates) caused a lack of air circulation, while there was a reluctance to install underground pipes for suitable waste disposal, since the could provide potential escape routes. There was constant criticism of conditions in the gaol in the late 19th century. The jail was built for 732 prisoners. The women's cell block was for 156 women. But in the 1850s there were as many as 450 women there. Over the years, a total of 79 people were executed at Darlinghurst Jail. It was used as a prison from 1840 till 1912.

In 1912 the gaol at Long Bay was complete and the Darlinghurst establishment was transferred to this site. The old gaol buildings were used as an internment camp during WWI. After consideration the site was transferred in 1921, to the department of Education which has adapted the building for the use of the East Sydney Technical College.

TAFE security guards are often confronted with spooky occurrences when doing their rounds of the buildings. One security guard that remains nameless said:

"It use to be really bad in here. What would happen was lights would come one and the doors would close. There was a smell too. It was so bad that you couldn't stay in the room. Maybe you could say it smelt like dirty socks or rats. We got the place cleaned and the lights checked but the problem was not fixed. I would race through here at night; I really did not like the feeling. I still don't like it a night. It was especially bad near the stairs, and it happened when there were not many people around. The ghost was a lady an Asian lady and she was looking for her husband who was a prisoner."

Near the staircase outside there are about five spirits, which can be heard in the blue room knocking on the blackboard. Another occurrence occurred when three security guards and were unloading fashion supplies for one of the rooms. After finishing the delivery the doors began closing behind them. Teachers have also been the victims of ghostly encounters with two teachers from catering and nutrition who have had run ins with spirits. One of the teachers studied Tai Chi and was a spiritual sort of person and he saw a ghost in the men's toilet in Building R (5). He claims there was just a head and a face, which was blue-grey and old. The teacher fell to the floor with a sickly feeling and felt drained till he left the room. There has also been another report of a spirit who followed that teacher home all the way to Glebe Island Bridge.