Since the sum of three internal angles of a triangle is equal to \pi

A+B+C=\pi

\cos C=-\cos(A+B)

Using double angle and sum to product identities

\cos^2A+\cos^2B+\cos^2C

\dfrac{\cos 2A+1}{2} +\dfrac{\cos 2B+1}{2} +\cos^2C

=1+\dfrac{\cos 2A+\cos 2B}{2} +\cos^2C

=1+\cos(A+B)\cos(A-B)++\cos^2C

=1+\cos C[(\cos C-\cos(A-B)]

=1-\cos C[\cos(A+B)+\cos(A-B)]

=1-2\cos A\cos B\cos C