They were ordinary people who fell on hard times, and they were buried in unmarked paupers' graves.
And though the site of the graveyard where more than 500 former Aylsham Workhouse inmates are interred has been left unloved and overgrown, it is now being transformed into a community garden and a space to reflect on the past.
Wendy Sadler, from Aylsham, is working on the project to restore the paupers' graves site at the workhouse, which later became St Michael's Hospital and was closed in 2012.
Mrs Sadler said: "The site has been derelict for over six years, so we've been doing some sessions to get ready for grass seed sowing and planting woodland plants.
"Volunteers are doing this with the co-operation with the town council, who are also in the process of putting in a path to make the area more accessible.
"This is about reclaiming it as part of Aylsham's history. When we're finished we aim to put a storyboard at the entrance to tell the history of the site to visitors."
As well as the town council, the project is supported by Aylsham in Bloom, the Royal Horticultural Society and grant from Greening Great Britain.
Aylsham Workhouse was built in 1848-49, just over a decade after the Aylsham Poor Law union was formed. People were usually sent there because they were too poor, old or ill to support themselves.
The graveyard was in use between 1857 and 1903 and Aylsham resident Diana Duhig has painstakingly recorded details of all 548 people buried there, using workhouse information kept in the Norfolk Record Office.
They range in age from one-day-old twins, Florence and Martha Garford, buried in January 1883, to someone in their mid 90s.
Volunteers can take part in garden clearing sessions on Monday, September 6 at 10am, September 7 at 2pm, September 8 at 10am and September 11 at 2pm. To get in touch with Mrs Sadler, email wmsadler42@gmail.com or call 078550 49963.
Mrs Sadler said: "We are making good progress but need more volunteers, so please come and join us. Bring a rake or hoe a fork, gardening gloves and wear stout shoes."
Comments: Our rules
We want our comments to be a lively and valuable part of our community - a place where readers can debate and engage with the most important local issues. The ability to comment on our stories is a privilege, not a right, however, and that privilege may be withdrawn if it is abused or misused.
Please report any comments that break our rules.
Read the rules here