If the tyres are too different front and back it can muck with your awd/4mo system (if you have one) and braking.
There is tolerance for both of these though.

On a 10 a 265/40 Will be pretty square.

A 10" wheel is actually 11" over all as you know, convert 11" to mm (279mm) this will give you some idea. Keep in mind you can get 'narrow' and 'wide' tyres in each size pending brand and model.

I've happily fit a 235/35 on my 10" wide wheels before and drive to Sydney and back no worries, but that's probably not what you want haha