Do you have metrics for when users stop using your apps? Numbers I saw a year ago said the vast majority of downloaded apps are run 0 or 1 times. So the biggest single hurdle is getting people to run your app two times. Design your app with that in mind.
Increasing downloads is hard, and many good apps fail. Apple and Google rely on developer churn – there will always be more people trying to upload apps, even though today’s app developers get burned out because of lack of money from the app stores. The stores don’t have much incentive to make new apps easy to find. If you rely on search engines for users to find your app, you’re probably going to be disappointed, You might go viral, but that’s extremely rare.
You need a plan to get more users. Work out that plan before you start coding. Or accept the fact that you are primarily coding for yourself, and maybe other people will like it too.