The plotly cell used in the logit lens tutorial https://nnsight.net/notebooks/tutorials/logit_lens/ has a bug:
If there are duplicate input_words, plotly removes the duplicates, which offsets the columns and (leads to confusing results)
adding the following snip-it solves the problem (though possible there are less verbose methods)
def make_unique_with_invisible_chars(strings):
seen = {}
result = []
for string in strings:
if string in seen:
seen[string] += 1
modified_string = string + '\u200B' * seen[string]
result.append(modified_string)
else:
seen[string] = 0
result.append(string)
return result
input_words = make_unique_with_invisible_chars(input_words)(original cell)
import plotly.express as px
import plotly.io as pio
if is_colab:
pio.renderers.default = "colab"
else:
pio.renderers.default = "plotly_mimetype+notebook_connected+notebook"
fig = px.imshow(
max_probs.detach().cpu().numpy(),
x=input_words,
y=list(range(len(words))),
color_continuous_scale=px.colors.diverging.RdYlBu_r,
color_continuous_midpoint=0.50,
text_auto=True,
labels=dict(x="Input Tokens", y="Layers", color="Probability")
)
fig.update_layout(
title='Logit Lens Visualization',
xaxis_tickangle=0
)
fig.update_traces(text=words, texttemplate="%{text}")
fig.show()
The plotly cell used in the logit lens tutorial https://nnsight.net/notebooks/tutorials/logit_lens/ has a bug:
If there are duplicate
input_words, plotly removes the duplicates, which offsets the columns and (leads to confusing results)adding the following snip-it solves the problem (though possible there are less verbose methods)
(original cell)