Fix invisible load more button (#4962)

* Fix behavior while the button is invisible
e.g. pointer cursor, couldn't open contextmenu
* Avoid rendering the button to remove blank space if no more items are available or no items are rendered