Solution 1 (Algebra: Generalized)
More generally, let (x,y,z)=(5,6,7) so that (x2,y2,z2)=(5,6,7).
We rewrite the original expression in terms of x,y, and z, then apply the difference of squares repeatedly:
(x+y+z)(x+y−z)(x−y+z)(−x+y+z)=[((x+y)+z)((x+y)−z)][((z+(x−y))(z−(x−y))]=[(x+y)2−z2][z2−(x−y)2]=[x2+2xy+y2−z2][z2−x2+2xy−y2]=[2xy+(x2+y2−z2)][2xy−(x2+y2−z2)]=(2xy)2−(x2+y2−z2)2=(2⋅5⋅6)2−(5+6−7)2=104.
Remark
From this solution, note that the original expression has cyclic symmetry with respect to x,y, and z:
(x+y+z)(x+y−z)(x−y+z)(−x+y+z)=⋯=(2xy)2−(x2+y2−z2)2=4x2y2−x4−y4−z4−2x2y2+2y2z2+2z2x2=2x2y2+2y2z2+2z2x2−x4−y4−z4.
~MRENTHUSIASM
Solution 2 (Algebra: Specific)
We group the first and last factors as well as the two middle factors, then apply the difference of squares repeatedly:
((6+7)2−52)(52−(6−7)2)=(13+242−5)(5−(13−242))=(242+8)(242−8)=(242)2−82=104.
~Azjps (Solution)
~MRENTHUSIASM (Revision)
Solution 3 (Geometry)
Notice that in a triangle with side-lengths 25,26, and 27, by Heron's Formula, the area is the square root of the original expression.
Let θ be the measure of the angle opposite the 27 side. By the Law of Cosines,
cosθ=2⋅25⋅26(25)2+(26)2−(27)2=83016=152,
so sinθ=1−cos2θ=1513.
The area of the triangle is then
2sinθ⋅25⋅26=3026⋅230=104,
so our answer is (104)2=104.