Public built up too much debt, admits minister