Changes to disability benefits are being "rushed through" to meet Treasury targets, campaigners have argued as peers prepare to debate the issue.
They want ministers to delay changes to Disability Living Allowance, saying new medical assessment tests are not ready.
Former RNIB chairman and crossbench peer Lord Low said "the livelihoods of disabled people were at stake".
But ministers said DLA was "20 years out of date" and £600m a year was being paid out to people no longer eligible.
A group of cross-bench peers, backed by some charities, want planned changes to Disability Living Allowance (DLA) and its replacement - the Personal Independence Payment (PIP) - to be put on hold pending further consultation.
Campaigners say the plans will drive more people into poverty. Ministers have conceded some changes but say their plans will focus help on those who need it most.
Introduced in 1992 to help disabled people cope with the extra costs they face in their daily lives, DLA is paid to two million people of working age. It is thought that half a million fewer people will qualify for PIPs by 2015 if the changes are passed.
One critic of the proposals, Lord Low - president of the Disability Alliance charity - said the changes were being driven by the government's need to save money rather than the interests of disabled people.
"This is being rushed through to meet Treasury targets," he told BBC Radio 4's Today programme.
"The assessment system for disabled people, for the new PIP benefit, is not ready yet. This is a work in progress. Disabled people do not have confidence that the government is yet ready to deliver an assessment system which will achieve the outcome which is fair to them and takes proper account of disabled people's needs for a benefit of this kind."
The problems with a new "work assessment" test for incapacity benefit - which is causing controversy years after coming into force - should be a warning to ministers, he added.
"We need to take more time to get it right as disabled people's livelihoods are at stake."
But disabilities minister Maria Miller said campaigners were wrong to suggest that DLA was being cut by 20% and the government remained "absolutely committed to the idea and practice" of enabling people with disabilities to live independent lives.
"What we are trying to do in these difficult economic times is to make sure that the rate of growth does not continue to spiral in the way as it has done in the past," she told the BBC.
"In the future we will be spending the same on DLA as we did last year."
'Benefit for life'
Many vulnerable people were currently "falling through the net", she added, and changes were needed to ensure support was "getting through to the people who need it the most".
"We know that DLA is not a modernised benefit, it does not support people with severe mental health problems and learning difficulties in the way we would want to in this day and age as it is a benefit which is 20 years out of date."
Ms Miller said flaws in the system also meant that £600m was being paid out every year to people whose conditions did not justify it while others were being under-supported when their medical situations worsened. "There is not in-built reassessment and that means 70% of people are claiming this benefit for life."
The government has already agreed to halve the time seriously ill or disabled people will have to wait to be eligible for PIPs from six to three months.
The move came after peers defeated the coalition over other proposed changes to eligibility for another benefit, employment support allowance (ESA) - formerly known as incapacity benefit.
The government wants to pass its welfare reform bill, one of its flagship pieces of legislation, by the end of parliamentary session in May.
It says the proposals will substantially reduce the multi-billion pound welfare bill, helping to cut the deficit while also increasing incentives to work and targeting support for the vulnerable more effectively.