MP4 is the popular name for the MPEG-4 standard, defined by the Moving Picture Experts Group, codified as ISO/IEC 14496. Saying just what the "MP4 file format" is, is a somewhat complicated question. Parts 12, 14, and 15 specifically refer to "file formats," but the term "MP4 format" is often taken to refer specifically to Part 10, which is a compression format and not a file format.
Part 12 is known as ISO Base Media File Format. It is derived largely from Apple's QuickTime format and is described as a "general format for timed sequences of media data."
Part 14 is an extension of Part 12. The recommended extension for it is ".mp4", though ".m4a" is often used for audio-only files.