Using completing the square process to find the maximum and minimum of the quadratic function, that is, converting the quadratic to a perfect square plus a constant

x^2-xy-y^2

=(x-\dfrac{1}{2}y)^2-\dfrac{5}{4}y^2 \geq -\dfrac{5}{4}y^2 \geq -\dfrac{5}{4}

On the other hand,

x^2-xy-y^2

=-(y+\dfrac{1}{2}x)^2+\dfrac{5}{4}x^2 \leq \dfrac{5}{4}x^2 \leq \dfrac{5}{4}

Therefore,

0 \leq |x^2-xy-y^2|\leq \dfrac{5}{4}