For a CNN Policy, since we will be mostly applying it for Gym Atari environments, would it be fine to take the settings as in the original DQN paper?

Yes. The architectures mentioned above are standard in RL

@threewisemonkeys-as You were working on Impala right? Any chance we could fit some of it in here?

I haven't added the CNN/LSTM policy to that yet, only built the actor / learner architecture so far with an Mlp policy here

Also what is the difference between Cnn LSTM (Nature) and IMPALA Cnn LSTM

I don't think we need any LSTM policies right now? Impala CNN would be nice for Atari tho.

Can I take this up?
@sampreet-arthi said that there is support missing on the envs that require LSTM policies.. but still.. making it complete might help
Same for CNNLSTM.. Not sure if I can do the Impala ones tho

It might be a better idea to work on #194 first. After which new policies will be easy to add

Would just like to point out my comment in #222. Should we focus more on making policies easily extendible? Instead of adding multiple policies and passing string arguments for each policy?

I guess its already implemented in #205 with support for custom policies, values and actorcritics