Cells must fine-tune their gene expression programs, for optimal cellular activities in their natural growth conditions. Transcriptional memory, a unique transcriptional response, plays a pivotal role in faster reactivation of genes upon environmental changes, and is facilitated if genes were previously in an active state. Hyper-activation of gene expression by transcriptional memory is critical for cellular differentiation, development, and adaptation. TREM (Transcriptional REpression Memory), a distinct type of transcriptional memory, promoting hyper-repression of unnecessary genes, upon environmental changes has been recently reported. These two transcriptional responses may optimize specific gene expression patterns, in rapidly changing environments. Emerging evidence suggests that they are also critical for immune responses. In addition to memory B and T cells, innate immune cells are transcriptionally hyperactivated by restimulation, with the same or different pathogens known as trained immunity. In this review, we briefly summarize recent progress in chromatin-based regulation of transcriptional memory, and its potential role in immune responses.