November's exam results show 22.9% of maths entries were marked at a grade 4, a standard pass, or above, down from 24.9% in 2022 and 26.9% in 2019.
However, the pass rate for GCSE English resits rose to 40.3% - up from 38% and 32.3% respectively.
Under-18s in England must retake GCSE English and maths if they did not get at least a grade 4.
The overall GCSE pass rate fell last summer, when grades in England were brought back in line with pre-pandemic levels.
Exams sat in November, which are typically resits, were marked on the same basis.
Ahead of the November resits, colleges told the BBC they were having to expand class sizes and hire exam halls to cope with a rising number of pupils taking compulsory GCSE resits.
Exams were cancelled in 2020 and 2021 and grades were based on teachers' assessments, leading to a rise in top results.
Grades were brought back to 2019 levels in England last summer - meaning a lower pass-rate, and, therefore, more students having to take compulsory resits.
More than 167,000 students in England received grade 3 or lower on their maths paper last summer - about 21,000 more than in 2022.
A further 172,000 failed English language - 38,000 more.
At the same time, there are more teenagers coming through the system.
Analysis by the Institute for Fiscal Studies suggests there will be a 17% rise in the number of 16 and 17-year-olds between 2019 and 2024 - an extra 200,000 young people.
Although the pass rate fell in maths, the raw number of passes in both maths and English grew, because more students sat November exams than in previous years.
There were 117,098 entries for maths and English in November - compared with 91,357 in 2022 and 102,960 in 2019.
Eddie Playfair, senior policy manager at the Association of Colleges, welcomed the rise in pass grades.
He said November candidates will be "only a subset of the total number resitting" - as students can also resit in the summer - and "will tend to be those most likely to improve their grade quickly".
England's exams regulator, Ofqual, said the return to "normal grading" last year meant it was "not meaningful to make direct comparisons between results in recent November series".
Comparisons with 2019 were included in the release of the November results - the last year exams were sat before the pandemic.
'Remorseless'
Tom Middlehurst, assessment specialist at the Association of School and College Leaders, said the compulsory resit policy "consigns the majority of young people to a remorseless cycle of falling below the grade 4 bar".
He said schools and colleges have tried to help students catch up on "lost learning" because of the Covid pandemic, but have "constrained resources".
The fall in the maths pass rate comes after the government announced plans to replace A-levels and T-levels with a new Advanced British Standard which would include some English and maths to 18.
A spokeswoman for the Department for Education (DfE) said: "Young people who leave education with a good grasp of English and maths have a much better chance of securing a job or going on to further study."
The DfE said it was investing an extra £1.6bn in colleges by 2024-25, compared with the 2021-22 financial year, and an extra £470m over this year and next would would help colleges boost recruitment and retention of staff.