How do I decrease loss on keras sequential model

Question

I am hoping to get some guidance on what steps I should take next in my attempt to model a certain system. It contains 3 independent variables, 24 dependent variables, and about 21,000 rows. In my attempts to model, I cannot get the accuracy above about 50% or the loss less than about 6500. I've been using variations on the following code:

EPOCHS = 30
#OPTIMIZER = 'adam'
#OPTIMIZER = 'adagrad'
BATCH_SIZE = 10
OUTPUT_UNITS = len(y.columns)
print(f'OUTPUT_UNITS: {OUTPUT_UNITS}')

model = Sequential()
model.add(Dense(8, activation='relu', input_dim=3)) # 3 X parameters, with eng_speed removed
#model.add(Dense(8, activation='relu', input_dim=4)) # 4 X parameters
model.add(Dense(32, activation='relu' ))
#model.add(Dense(64, activation='relu' ))
#model.add(Dense(12, activation='relu' ))
model.add(Dense(OUTPUT_UNITS)) # number of predicted (y) values (labels)
model.summary()

adadelta = optimizers.Adadelta()
adam = optimizers.Adam(lr=0.001)

model.compile(optimizer=adadelta, loss='mse', metrics=['accuracy'])
#model.compile(optimizer=opt, loss='mse', metrics=['accuracy'])
history = model.fit(x=X_train, y=y_train, epochs=EPOCHS, batch_size=BATCH_SIZE)

I've tried removing and adding layers, changing the size of them, different optimizers, learning rates, etc. The following two graphs are typical of what I've been seeing--they both flatten very quickly, then don't improve:

I'm obviously new at this and would appreciate it if someone pointed me in the right direction: an approach to try, something to read up on, whatever. Thanks in advance.

Answer 1

Since (according to your mse loss and your regression tag) you are in a regression setting, accuracy is meaningless (it is only used in classification settings); please see own answer in What function defines accuracy in Keras when the loss is mean squared error (MSE)?

Given that, there is in principle absolutely no reason to consider a loss of 6500 as "high", and hence needing improvement...