Considering the history of the racial slur in question, I wouldn't hold it against a black person to be offended by the use of the word, particularly by a white person. It's not like it's use as a derogatory term is non-existent anymore, it is still used as a racial slur among some people. Even if it's a term of endearment between black people, the connotation between a black person calling another black person that term is significantly different from a white person calling a black person that term. It's still a racial slur in the American lexicon, and it's only been a couple decades since systematic racism was "officially" outlawed.

Besides, it's not like it's a particularly special word to begin with. Any context where you would use it as a term of endearment can be replaced with a different word anyways.

I personally am not comfortable with any racial slurs, and I will make my discomfort on the use of those words clear when they are used, regardless of who uses them. Personal experience as a child experiencing teasing for my Asian heritage, and outright discrimination for my mixed background, gives me a low tolerance for any use of racial slurs.