The Blues were drawn at home to the Championship side, currently 20th in the table.
However, Paul Warne's side have been in fine form of late, losing just once in eight games and playing some excellent football in the process.
The teams have only met once before in the competition - back in 1979 when the Millers pulled off a shock 0-0 draw at Maine Road.
City then travelled to Millmoor for the replay and won 4-2.
Connections include Shaun Goater, Gerry Gow and Nigel Johnson.
The teams have only met 11 times competitively and the Blues have never lost.
Though Rotherham and City drew 1-1 in the Football League Trophy last season, the last meeting in the league was back in 2002, with Ali Benarbia scoring in a 1-1 draw.
The tie will be played over the weekend of January 4, 5, 6 and 7th with the date, time and ticket details to be confirmed in due course.
Bolton vs Walsall or Sunderland
Millwall vs Hull City
Gillingham vs Cardiff
Brentford vs Oxford United
Sheffield Wednesday vs Luton Town
Manchester United vs Reading
Everton vs Lincoln City
Tranmere or Southport vs Tottenham
Preston vs Doncaster
Newcastle vs Blackburn
Chelsea vs Nottingham Forest
Crystal Palace vs Grimsby
Derby vs Southampton
Accrington Stanley vs Ipswich
Bristol City vs Huddersfield
Wrexham or Newport vs Leicester
Fulham vs Oldham
Shrewsbury vs Stoke
Solihull or Blackpool vs Arsenal
Manchester City vs Rotherham
Bournemouth vs Brighton
West Ham vs Birmingham
Woking vs Watford
Burnley vs Barnsley
QPR vs Leeds
Sheffield United vs Barnet
Norwich vs Portsmouth
Guisley or Fleetwood vs AFC Wimbledon
West Brom vs Wigan
Middlesbrough vs Peterborough or Bradford
Wolves vs Liverpool
Aston Villa vs Swansea