If time were running against the arrow of entropy, nobody could perceive or measure it, right?
Remembering something is per se an increase in entropy, so the universe could run in negative time direction, but we would simply forget, what had happened.
That said, I personally think such thought experiments are futile and the nature of time has to be understood by its connection to causality and information.
Of course you could perceive it, measure it and record it. The entropy of your body or your brain is not necessarily increasing, nor is the entropy of your computer or other information storage systems.
Entropy only statistically tends towards an increase in closed systems and neither your computer or your brain are closed systems. They are both constantly getting energy from an external source of power and in turn dispersing previously consumed energy out into their environment.
And yet you still manage to perceive things just fine... in fact your perception of the world is unlikely to change whether or not the entropy in your brain increases or decreases by some bounded amount (of course too much of either an increase or decrease will destroy your brain).
Your claim about remembering an event, which likely alludes to Laplace's demon [1], requires an overall increase in entropy in the system as a whole, but does not require an increase in entropy in the specific part of the system that is recording the event.
Every time your computer calls a function like memset(dst, 0), or sorts a list, or arranges data into some kind of structured binary tree, your computer is decreasing its own internal entropy by taking a statistically likely arrangement of bits and transforming it into a very unlikely arrangement of bits. The decrease in the internal entropy of your computer is more than offset by an increase in global entropy but that global entropy is radiating way out into the cosmos and has no impact on your computer's ability to register information.
That said, I personally think such thought experiments are futile and the nature of time has to be understood by its connection to causality and information.