The Dr. Mariano Elia Hands on Center at the Art Gallery of Ontario is a terrific place to take kids. As you can see in the pictures there are a ton of mostly arts and crafts-centric games and supplies for kids to play/be creative with.
We have gone three times. The prior two times the kids were mostly indifferent to it. We lasted at most an hour. But this time, at ages 3 and almost 5 we hit the sweet spot as the kids loved it and we spent the entire Sunday afternoon there.
AGO tickets can be crazy expensive but most libraries in the city give out free family passes every Saturday morning. We’ve never had problems getting free passes. The Centre is in the basement of the AGO which is on Dundas, a block west of University. If you go by subway get off at the St. Patrick subway station which, sadly, does not yet have an elevator for strollers. (see map below)
There is also a kids gallery upstairs (pictured with the green table below) where kids can also draw. Not as fun but still worth the visit on the way out.
All in all, if your kids are into it, it can be a wonderful day.