Ta có:

a)

 Có $\overrightarrow{AB} . \overrightarrow{AC} = AB . AC . cos\widehat{CAB} = 4.5.cos60^{o} = 10$

$\overrightarrow{AB} . \overrightarrow{AC} = \overrightarrow{AB} ( \overrightarrow{AC} - \overrightarrow{AB}) = \overrightarrow{AB} . \overrightarrow{AC} - \overrightarrow{AB}^{2} = 10 - 4^{2} = -6$

b) 

Có $2\overrightarrow{AM} + 3\overrightarrow{MC} = \overrightarrow{0}$

$<=> 2(\overrightarrow{AB} + \overrightarrow{BM}) + 3(\overrightarrow{BC} - \overrightarrow{BM} = \overrightarrow{0}$

$<=> \overrightarrow{BM} = -2\overrightarrow{AB} - 3\overrightarrow{BC} = 2\overrightarrow{AB} + 3(\overrightarrow{AC} - \overrightarrow{AB}) = -\overrightarrow{AB} + 3\overrightarrow{AC}$ (1)

Có $<=>{NB} + x\overrightarrow{NC} = \overrightarrow{0}$

$<=> (\overrightarrow{AB} - \overrightarrow{AN}) + x(\overrightarrow{AC} - \overrightarrow{AN}) = \overrightarrow{0}$

$<=> (1 + x)\overrightarrow{AN} = \overrightarrow{AB} + x\overrightarrow{AC}$ (2)

Từ (1) và (2) 

=> $(1 + x)\overrightarrow{AN} . \overrightarrow{BM} = (\overrightarrow{AB} + x\overrightarrow{AC}) (-\overrightarrow{AB} + 3\overrightarrow{AC})$

$<=> (1 + x)\overrightarrow{AN} . \overrightarrow{BM} = -\overrightarrow{AB}^{2} +3\overrightarrow{AB}.\overrightarrow{AC} - x\overrightarrow{AC}.\overrightarrow{AB} + 3x\overrightarrow{AC}^{2}$

$<=> (1 + x)\overrightarrow{AN} . \overrightarrow{BM} = -16 + 3.10 - x.10 + 3x.25 = 65x + 14$

Để AN $\perp$ BM $\Leftrightarrow \overrightarrow{AN} . \overrightarrow{BM} = 0$

$<=> 65x + 14 = 0$

$<=> x = \frac{-16}{64}$ (thỏa mãn)