Usually it comes down to the "definition of gin having its main characteristic flavor from juniper berries". If you don't use a neutral base, the primary characteristic would change to that of the base itself.
If for instance you used corn, it would end up being a corn whiskey with botanicals and be considered a "distilled speciality spirit".
There is a class of gin called "Genever" this is different from the gins as defined in the TTB chapter 4, it defined as "a clear, botanically rich, malted grain-based spirit that can only be made in Holland or Belgium"
So you could make a Genever style gin, in the method of what you mentioned with a lower proof base.