Make git NOT store object files in the .git directory

I would like to use git to track change history messages and hashes of files to track changes, but I don't want it to store the objects in the .git directory.

Why? Because putting 4TB in a git repository and changing things is terrible when it stores everything. However, I want tracking and metadata for history, changes, and its help doing synchronization and conflict detection.

Can git do this natively? Do I have to man-handle it by telling it to remove objects?