The government expects to raise £3 billion a year through the levy, which will be imposed from April 2017.
Only businesses with a wage bill of more than £3m will pay it, which the government said would exempt 98% of employers.
The charge will help ensure that big business shoulders the cost of training workers, George Osborne added.
He told the Commons: "It's a huge reform to raise the skills of the nation and address one of the enduring weaknesses of the British economy."
But concerns have been raised by some employers that the combination of the new national living wage with increased compulsory investment in training would mean budgets elsewhere in organisations could be squeezed.
The Office for Budget Responsibility has warned that the apprenticeship levy will damage wage growth as employers pass on the cost to their employees. It said the levy, together with the cost of pensions auto-enrolment, would reduce cumulative wage growth over the next five years by 0.7 per cent.
Business groups have also described the levy as a new "payroll tax". The Institute of Directors said it would be a big cost for many companies, including medium-sized firms.
Carolyn Fairbairn, the new director-general of the CBI, called the apprenticeship levy a “sting in the tail” of the Spending Review for employers.
She said: “It is a significant extra payroll tax on business and by widening the net it will now catch more smaller firms. You only need to have 100 to 150 staff on minimum wage, so quite a few small firms will be caught by this.”
The Government has also announced that it will establish a new, employer-led body to set apprenticeship standards and ensure quality.
">