Sitelet https://web.archive.org/web/20210715102704/https://github.com/pytorch/ignite/issues/2100
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update BaseOutputHandler to accept state attributes #2100

Open
8 tasks
vfdev-5 opened this issue Jul 6, 2021 · 0 comments
Open
8 tasks

Update BaseOutputHandler to accept state attributes #2100

vfdev-5 opened this issue Jul 6, 2021 · 0 comments

Comments

@vfdev-5
Copy link
Task lists! Give feedback
Collaborator

@vfdev-5 vfdev-5 commented Jul 6, 2021

🚀 Feature

Here is the feature I would like to have, for example, with TensorboardLogger, but we can implement that for all exp tracking systems:

tb_logger = TensorboardLogger(log_dir="experiments/tb_logs")

tb_logger.attach_output_handler(
    trainer,
    event_name=Events.ITERATION_COMPLETED,
    tag="training",
    state_attrs=["alpha", "beta"]  # <->  trainer.state.alpha and trainer.state.beta
)

where state_attrs defines some attributes from trainer.state to log to TB.

Currently, this could be done as

tb_logger = TensorboardLogger(log_dir="experiments/tb_logs")

def trainer_state_logger(state_attrs, tag=None):

    def wrapper(engine, logger, event_name):
        assert engine == trainer

        global_step = engine.state.get_event_attrib_value(event_name)
        tag_prefix = f"{tag}/" if tag else ""
        for name in state_attrs:
            value = getattr(engine, name, None)
            if value is None:
                continue
            logger.writer.add_scalar(f"{tag_prefix}{name}", value, global_step)
    
tb_logger.attach(
    trainer,
    event_name=Events.ITERATION_COMPLETED,
    log_handler=trainer_state_logger(["alpha", "beta"], tag="training")
)

We would like to support state attributes types same as for metrics:
See

for key, value in metrics.items():
if isinstance(value, numbers.Number):
logger.writer.add_scalar(f"{self.tag}/{key}", value, global_step)
elif isinstance(value, torch.Tensor) and value.ndimension() == 0:
logger.writer.add_scalar(f"{self.tag}/{key}", value.item(), global_step)
elif isinstance(value, torch.Tensor) and value.ndimension() == 1:
for i, v in enumerate(value):
logger.writer.add_scalar(f"{self.tag}/{key}/{i}", v.item(), global_step)
else:
warnings.warn(f"TensorboardLogger output_handler can not log metrics value type {type(value)}")

This issues can be handled in several PRs that should implement the feature for the following exp tracking systems and loggers:

  • TensorBoard logger
  • Visdom logger
  • ClearML logger
  • Neptune logger
  • WandB logger
  • MLFlow logger
  • Polyaxon logger
  • tqdm logger

If you would like work on this issue, please comment out and say which part you would like to cover.
A PR should contain new code, tests and documentation updates. Please see our contributing guide: https://github.com/pytorch/ignite/blob/master/CONTRIBUTING.md
A draft PR can be sent and maintainers can help to iterate over the remaining tasks.

@vfdev-5 vfdev-5 added this to To do in 0.5.1 via automation Jul 6, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
0.5.1
  
To do
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
1 participant