Question:
Which techniques are key components of the Kg-GPT framework for enhancing reasoning over knowledge graphs?
A. Structured prompting
B. Adaptive retrieval mechanisms
C. Graph-based data augmentation
D. Incorporation of graph structure embeddings into LLM inputs

Answer: AB

KG-GPT consists of three steps: 1) Sentence (Claim / Question) Segmentation, 2) Graph Retrieval, and 3) Inference. During Sentence Segmentation, a sentence is partitioned into discrete sub-sentences, each aligned with a single triple (i.e. [head, relation, tail]). The subsequent step, namely Graph Retrieval, retrieves a potential pool of relations that could bridge the entities identified within the sub-sentences. Then, a candidate pool of evidence graphs (i.e. sub-KG) is obtained using the retrieved relations and the entity set. In the final step, the obtained graphs are used to derive a logical conclusion, such as validating a given claim or answering a given question.

We assume a graph G (knowledge graph consisting of entities E and relations R), a sentence S (claim or question), and all entities involved in S, ES ⊂E are given. In order to derive a logical conclusion, we need an accurate evidence graph GE ⊂G, which we obtain in two stages, Sentence Segmentation and Graph Retrieval. Furthermore, all the aforementioned steps are executed employing the in-context learning methodology to maximize the LLM’s reasoning ability. The prompts used for each stage are in Appendix A.

2.1 Sentence Segmentation
Many KG-based tasks require multi-hop reasoning. To address this, we utilize a Divide-and-Conquer approach. By breaking down a sentence into subsentences that correspond to a single relation, identifying relations in each sub-sentence becomes easier than finding n-hop relations connected to an entity from the original sentence all at once.
We assume S can be broken down into sub-sentences: S1, S2, ..., Sn where Si consists of a set of entities Ei ⊂E and a relation ri ∈R. Each e(j)i ∈Ei can be a concrete entity (e.g. William Anders in Fig. 1-(1)), or a type (e.g. artificial satellite in Fig. 1-(1)). ri can be mapped to one or more items in R, as there can be multiple relations with similar semantics (e.g. birthPlace, placeOfBirth).

2.2 Graph Retrieval
To effectively validate a claim or answer a question, it is crucial to obtain an evidence graph (i.e. sub-KG) that facilitates logical conclusions. In this stage, we first aim to retrieve the corresponding relations for each sub-sentence Si to extract GE. For each Si, we use the LLM to map ri to one or more items in R as accurately as possible. To do so, we first define Ri ⊂R, which is a set of relations connected to all e(j)i ∈Ei according to the schema of G (i.e. relation candidates in Fig. 1-(2)). This process considers both the relations connected to a specific entity and the relations associated with the entity’s type in G. We further elaborate on the process in Appendix B. Then, we feed Si and Ri to the LLM to retrieve the set of final top-K relations Ri,k. In detail, relations in Ri are linearized (e.g. [location, birthYear, ..., birthDate]) and combined with the corresponding sub-sentence Si to establish prompts for the LLM and the LLM generates Ri,k = {r(1)i, ..., r(k)i} as output. In the final graph retrieval step, we can obtain GE, made up of all triples whose relations come from Ri,k and whose entities come from Ei across all Si.

2.3 Inference
Then, we feed S and GE to the LLM to derive a logical conclusion. In order to represent GE in the prompt, we linearize the triples associated with GE (i.e. [[head1, rel1, tail1], ..., [headm, relm, tailm]]), and then concatenate these linearized triples with the sentence S. In fact verification, the determination of whether S is supported or refuted is contingent upon GE. In question answering, the LLM identifies an entity in GE as the most probable answer to S.

Findings of the Association for Computational Linguistics: EMNLP 2023, pages 9410–9421
December 6-10, 2023 ©2023 Association for Computational Linguistics
KG-GPT: A General Framework for Reasoning on Knowledge Graphs
Using Large Language Models
Jiho Kim1, Yeonsu Kwon1, Yohan Jo2, Edward Choi1
1KAIST 2Seoul National University
{jiho.kim, yeonsu.k, edwardchoi}@kaist.ac.kr
yohan.jo@snu.ac.kr
Abstract
While large language models (LLMs) have
made considerable advancements in under-
standing and generating unstructured text, their
application in structured data remains underex-
plored. Particularly, using LLMs for complex
reasoning tasks on knowledge graphs (KGs)
remains largely untouched. To address this,
we propose KG-GPT, a multi-purpose frame-
work leveraging LLMs for tasks employing
KGs. KG-GPT comprises three steps: Sentence
Segmentation, Graph Retrieval, and Inference,
each aimed at partitioning sentences, retriev-
ing relevant graph components, and deriving
logical conclusions, respectively. We evalu-
ate KG-GPT using KG-based fact verification
and KGQA benchmarks, with the model show-
ing competitive and robust performance, even
outperforming several fully-supervised models.
Our work, therefore, marks a significant step
in unifying structured and unstructured data
processing within the realm of LLMs.1
1
Introduction
The remarkable advancements in large language
models (LLMs) have notably caught the eye of
scholars conducting research in the field of natural
language processing (NLP) (Brown et al., 2020;
Chowdhery et al., 2022; OpenAI, 2023a,b; Anil
et al., 2023). In their endeavor to create LLMs
that can mirror the reasoning capabilities inherent
to humans, past studies have primarily centered
their attention on unstructured textual data. This
includes, but is not limited to, mathematical word
problems (Miao et al., 2020; Cobbe et al., 2021;
Patel et al., 2021), CSQA (Talmor et al., 2019), and
symbolic manipulation (Wei et al., 2022). While
significant strides have been made in this area, the
domain of structured data remains largely unex-
plored.
1Code is available at https://github.com/jiho283/
KG-GPT.
Structured data, particularly in the form of
knowledge graphs (KGs), serves as a reservoir of in-
terconnected factual information and associations,
articulated through nodes and edges. The inherent
structure of KGs offers a valuable resource that
can assist in executing complex reasoning tasks,
like multi-hop inference. Even with these advan-
tages, to the best of our knowledge, there is no
general framework for performing KG-based tasks
(e.g. question answering, fact verification) using
auto-regressive LLMs.
To this end, we propose a new general frame-
work, called KG-GPT, that uses LLMs’ reasoning
capabilities to perform KG-based tasks. KG-GPT
is similar to StructGPT (Jiang et al., 2023) in that
both reason on structured data using LLMs. How-
ever, unlike StructGPT which identifies paths from
a seed entity to the final answer entity within KGs,
KG-GPT retrieves the entire sub-graph and then
infers the answer. This means KG-GPT can be
used not only for KGQA but also for tasks like
KG-based fact verification.
KG-GPT consists of three steps: 1) Sentence
(Claim / Question) Segmentation, 2) Graph Re-
trieval, and 3) Inference. During Sentence Seg-
mentation, a sentence is partitioned into discrete
sub-sentences, each aligned with a single triple (i.e.
[head, relation, tail]). The subsequent step, namely
Graph Retrieval, retrieves a potential pool of rela-
tions that could bridge the entities identified within
the sub-sentences. Then, a candidate pool of evi-
dence graphs (i.e. sub-KG) is obtained using the
retrieved relations and the entity set. In the final
step, the obtained graphs are used to derive a logi-
cal conclusion, such as validating a given claim or
answering a given question.
To evaluate KG-GPT, we employ KG-based
fact verification and KGQA benchmarks, both de-
manding complex reasoning that utilizes structured
knowledge of KGs. In KG-based fact verification,
we use FACTKG (Kim et al., 2023), which includes
9410
Figure 1: An overview of KG-GPT. The framework comprises three distinct phases: Sentence Segmentation,
Graph Retrieval, and Inference. The given example comes from FACTKG. It involves a 2-hop inference from
‘William_Anders’ to ‘Frank_Borman’, requiring verification through an evidence graph consisting of three triples.
Both ‘William_Anders’ and ‘Frank_Borman’ serve as internal nodes in DBpedia (Lehmann et al., 2015), while
“AFIT, M.S. 1962” acts as a leaf node. Moreover, artificial satellite represents the Type information absent from the
provided entity set.
various graph reasoning patterns, and KG-GPT
shows competitive performance compared to other
fully-supervised models, even outperforming some.
In KGQA, we use MetaQA (Zhang et al., 2018), a
QA dataset composed of 1-hop, 2-hop, and 3-hop
inference tasks. KG-GPT shows performance com-
parable to fully-supervised models. Notably, the
performance does not significantly decline with the
increase in the number of hops, demonstrating its
robustness.
2
Method
KG-GPT is composed of three stages: Sentence
Segmentation, Graph Retrieval, and Inference, as
described in Fig. 1.
We assume a graph G (knowledge graph con-
sisting of entities E and relations R), a sentence
S (claim or question), and all entities involved in
S, ES ⊂E are given. In order to derive a logical
conclusion, we need an accurate evidence graph
GE ⊂G, which we obtain in two stages, Sentence
Segmentation and Graph Retrieval. Furthermore,
all the aforementioned steps are executed employ-
ing the in-context learning methodology to max-
imize the LLM’s reasoning ability. The prompts
used for each stage are in Appendix A.
2.1
Sentence Segmentation
Many KG-based tasks require multi-hop reasoning.
To address this, we utilize a Divide-and-Conquer
approach. By breaking down a sentence into sub-
sentences that correspond to a single relation, iden-
tifying relations in each sub-sentence becomes eas-
ier than finding n-hop relations connected to an
entity from the original sentence all at once.
We assume S can be broken down into sub-
sentences: S1, S2, ..., Sn where Si consists of a
set of entities Ei ⊂E and a relation ri ∈R. Each
e(j)
i
∈Ei can be a concrete entity (e.g. William An-
ders in Fig. 1-(1)), or a type (e.g. artificial satellite
in Fig. 1-(1)). ri can be mapped to one or more
items in R, as there can be multiple relations with
similar semantics (e.g. birthPlace, placeOfBirth).
2.2
Graph Retrieval
To effectively validate a claim or answer a question,
it is crucial to obtain an evidence graph (i.e. sub-
9411
KG) that facilitates logical conclusions. In this
stage, we first aim to retrieve the corresponding
relations for each sub-sentence Si to extract GE.
For each Si, we use the LLM to map ri to one
or more items in R as accurately as possible. To
do so, we first define Ri ⊂R, which is a set of
relations connected to all e(j)
i
∈Ei according to the
schema of G (i.e. relation candidates in Fig. 1-(2)).
This process considers both the relations connected
to a specific entity and the relations associated with
the entity’s type in G. We further elaborate on the
process in Appendix B. Then, we feed Si and Ri to
the LLM to retrieve the set of final top-K relations
Ri,k. In detail, relations in Ri are linearized (e.g.
[location, birthYear, ..., birthDate]) and combined
with the corresponding sub-sentence Si to estab-
lish prompts for the LLM and the LLM generates
Ri,k = {r(1)
i
, ..., r(k)
i
} as output. In the final graph
retrieval step, we can obtain GE, made up of all
triples whose relations come from Ri,k and whose
entities come from Ei across all Si.
2.3
Inference
Then, we feed S and GE to the LLM to derive a
logical conclusion. In order to represent GE in the
prompt, we linearize the triples associated with
GE (i.e. [[head1, rel1, tail1], ..., [headm, relm,
tailm]]), and then concatenate these linearized
triples with the sentence S. In fact verification, the
determination of whether S is supported or refuted
is contingent upon GE. In question answering, the
LLM identifies an entity in GE as the most probable
answer to S.
3
Experiments
We evaluate our framework on two tasks that re-
quire KG grounding: fact-verification and question-
answering. A detailed description of experimental
settings can be found in Appendix C.
3.1
Dataset
3.1.1
FACTKG
FACTKG (Kim et al., 2023) serves as a practical
and challenging dataset meticulously constructed
for the purpose of fact verification, employing a
knowledge graph for validation purposes. It encom-
passes 108K claims that can be verified via DBpe-
dia (Lehmann et al., 2015), which is one of the
available comprehensive knowledge graphs. These
claims are categorized as either Supported or Re-
futed. FACTKG embodies five diverse types of
reasoning that represent the intrinsic characteris-
tics of the KG: One-hop, Conjunction, Existence,
Multi-hop, and Negation. To further enhance its
practical use, FACTKG integrates claims in both
colloquial and written styles. Examples of claims
from FACTKG can be found in Appendix D.
3.1.2
MetaQA
MetaQA (Zhang et al., 2018) is a carefully curated
dataset intended to facilitate the study of question-
answering that leverages KG-based approaches in
the field of movies. The dataset encompasses over
400K questions, including instances of 1-hop, 2-
hop, and 3-hop reasoning. Additionally, it covers a
diverse range of question styles. Examples of ques-
tions from MetaQA can be found in Appendix D.
3.2
Baselines
For evaluation on FACTKG, we use the same base-
lines as in Kim et al. (2023). These baselines are
divided into two distinct categories: Claim Only
and With Evidence. In the Claim Only setting, the
models are provided only with the claim as their
input and predict the label. For this setting, in ad-
dition to the existing baselines, we implement a
12-shot ChatGPT (OpenAI, 2023b) baseline. In
the With Evidence scenario, models consist of an
evidence graph retriever and a claim verification
model. We employ the KG version of GEAR (Zhou
et al., 2019) as a fully supervised model.
In our exploration of MetaQA, we use a selection
of prototypical baselines well-known in the field
of KGQA. These include models such as KV-Mem
(Xu et al., 2019), GraftNet (Sun et al., 2018), Em-
bedKGQA (Saxena et al., 2020), NSM (He et al.,
2021), and UniKGQA (Jiang et al., 2022), which
operate in a fully supervised fashion. Additionally,
we implement a 12-shot ChatGPT baseline.
4
Results & Analysis
4.1
FACTKG
We evaluated the models’ prediction capability for
labels (i.e. Supported or Refuted) and presented
the accuracy score in Table 1. As a result, KG-GPT
outperforms Claim Only models BERT, BlueBERT,
and Flan-T5 with performance enhancements of
7.48%, 12.75%, and 9.98% absolute, respectively.
It also outperforms 12-shot ChatGPT by 4.20%.
These figures emphasize the effectiveness of our
framework in extracting the necessary evidence for
claim verification, highlighting the positive impact
9412
Input Type
Training
Strategy
Methods
Accuracy
Claim Only
full
BERT
65.20
BlueBERT
59.93
zero-shot
Flan-T5
62.70
12-shot
ChatGPT
68.48
With Evidence
full
GEAR
77.65
12-shot
KG-GPT
72.68
8-shot
67.68
4-shot
59.53
Table 1: The performance of the models on FACTKG.
Except for ChatGPT and KG-GPT, all performances are
obtained from Kim et al. (2023).
Training
Methods
MetaQA
MetaQA
MetaQA
Strategy
1-hop
2-hop
3-hop
full
KV-Mem
96.2
82.7
48.9
GraftNet
97.0
94.8
77.7
EmbedKGQA
97.5
98.8
94.8
NSM
97.1
99.9
98.9
UniKGQA
97.5
99.0
99.1
12-shot
ChatGPT
60.0
23.0
38.7
12-shot
96.3
94.4
94.0
8-shot
KG-GPT
95.8
93.8
68.8
4-shot
94.7
92.8
46.6
Table 2: The performance of the models on MetaQA
(Hits@1). The best results for each task and those of
12-shot KG-GPT are in bold.
of the sentence segmentation and graph retrieval
stages. The qualitative results including the graphs
retrieved by KG-GPT are in Appendix E.1.
Nonetheless, when compared to GEAR, a fully
supervised model built upon KGs, KG-GPT ex-
hibits certain limitations. KG-GPT achieves an ac-
curacy score of 72.68%, which is behind GEAR’s
77.65%. This performance gap illustrates the ob-
stacles encountered by KG-GPT in a few-shot sce-
nario, namely the difficulty in amassing a suffi-
cient volume of information from the restricted
data available. Hence, despite the notable progress
achieved with KG-GPT, there is clear room for im-
provement to equal or surpass the performance of
KG-specific supervised models like GEAR.
4.2
MetaQA
The findings on MetaQA are presented in Table 2.
The performance of KG-GPT is impressive, scoring
96.3%, 94.4%, and 94.0% on 1-hop, 2-hop, and 3-
hop tasks respectively. This demonstrates its strong
ability to generalize from a limited number of ex-
amples, a critical trait when handling real-world
applications with varying degrees of complexity.
Stage
FactKG
MetaQA
MetaQA
MetaQA
1-hop
2-hop
3-hop
Sentence
39
3
63
100
Segmentation
Graph Retrieval
17
4
3
0
Inference
44
93
34
0
Table 3: Number of errors from 100 incorrect samples
across each dataset.
Interestingly, the performance of KG-GPT
closely matches that of a fully-supervised model.
Particularly, it surpasses KV-Mem by margins of
0.1%, 11.7%, and 45.1% across three distinct tasks
respectively, signifying its superior performance.
While the overall performance of KG-GPT is simi-
lar to that of GraftNet, a noteworthy difference is
pronounced in the 3-hop task, wherein KG-GPT
outperforms GraftNet by 16.3%. The qualitative
results including the graphs retrieved by KG-GPT
are in Appendix E.2.
4.3
Error Analysis
In both FACTKG and MetaQA, there are no cor-
responding ground truth graphs containing seed
entities. This absence makes a quantitative step-by-
step analysis challenging. Therefore, we carried
out an error analysis, extracting 100 incorrect sam-
ples from each dataset: FACTKG, MetaQA-1hop,
MetaQA-2hop, and MetaQA-3hop. Table 3 shows
the number of errors observed at each step. No-
tably, errors during the graph retrieval phase are
the fewest among the three steps. This suggests
that once sentences are correctly segmented, iden-
tifying relations within them becomes relatively
easy. Furthermore, a comparative analysis between
MetaQA-1hop, MetaQA-2hop, and MetaQA-3hop
indicates that as the number of hops increases, so
does the diversity of the questions. This heightened
diversity in turn escalates the errors in Sentence
Segmentation.
4.4
Ablation Study
4.4.1
Number of In-context Examples
The results for the 12-shot, 8-shot, and 4-shot
from the FACTKG and MetaQA datasets are re-
ported in Table 1 and Table 2, respectively. Though
there was a predicted improvement in performance
with the increase in the number of shots in both
FACTKG and MetaQA datasets, this was not uni-
formly observed across all scenarios.
Notably,
MetaQA demonstrated superior performance, ex-
9413
ceeding 90%, in both the 1-hop and 2-hop scenar-
ios, even with a minimal set of four examples. In
contrast, in both the FACTKG and MetaQA 3-hop
scenarios, the performance of the 4-shot learning
scenario was similar to that of the baselines which
did not utilize graph evidence. This similarity sug-
gests that LLMs may struggle to interpret com-
plex data features when equipped with only four
shots. Thus, the findings highlight the importance
of formulating in-context examples according to
the complexity of the task.
4.4.2
Top-K Relation Retrieval
Table 11 shows the performance according to the
value of k in FACTKG. As a result, performance
did not significantly vary depending on the value of
k. Table 12 illustrates the average number of triples
retrieved for both supported and refuted claims, de-
pending on k. Despite the increase in the number
of triples as the value of k grows, it does not im-
pact the accuracy. This suggests that the additional
triples are not significantly influential.
In MetaQA, the performance and the average
number of retrieved triples are also depicted in
Table 13 and Table 14, respectively. Unlike the
FACTKG experiment, as the value of k rises in
MetaQA, it appears that more significant triples are
retrieved, leading to improved performance.
5
Conclusion
We suggest KG-GPT, a versatile framework that
utilizes LLMs for tasks that use KGs. KG-GPT is
divided into three stages: Sentence Segmentation,
Graph Retrieval, and Inference, each designed for
breaking down sentences, sourcing related graph
elements, and reaching reasoned outcomes, respec-
tively. We assess KG-GPT’s efficacy using KG-
based fact verification and KGQA metrics, and
the model demonstrates consistent, impressive re-
sults, even surpassing a number of fully-supervised
models. Consequently, our research signifies a sub-
stantial advancement in combining structured and
unstructured data management in the LLMs’ con-
text.
Limitations
Our study has two key limitations. Firstly, KG-GPT
is highly dependent on in-context learning, and its
performance varies significantly with the number
of examples provided. The framework struggles
particularly with complex tasks when there are in-
sufficient or low-quality examples. Secondly, de-
spite its impressive performance in fact-verification
and question-answering tasks, KG-GPT still lags
behind fully supervised KG-specific models. The
gap in performance highlights the challenges faced
by KG-GPT in a few-shot learning scenario due to
limited data. Future research should focus on op-
timizing language models leveraging KGs to over-
come these limitations.
Acknowledgements
This work was supported by Institute of Informa-
tion & Communications Technology Planning &
Evaluation (IITP) grant (No.2019-0- 00075), Na-
tional Research Foundation of Korea (NRF) grant
(NRF-2020H1D3A2A03100945), and the Korea
Health Industry Development Institute (KHIDI)
grant (No.HR21C0198), funded by the Korea gov-
ernment (MSIT, MOHW).
References
Rohan Anil, Andrew M Dai, Orhan Firat, Melvin John-
son, Dmitry Lepikhin, Alexandre Passos, Siamak
Shakeri, Emanuel Taropa, Paige Bailey, Zhifeng
Chen, et al. 2023. Palm 2 technical report. arXiv
preprint arXiv:2305.10403.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie
Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind
Neelakantan, Pranav Shyam, Girish Sastry, Amanda
Askell, et al. 2020. Language models are few-shot
learners. Advances in neural information processing
systems, 33:1877–1901.
Aakanksha Chowdhery, Sharan Narang, Jacob Devlin,
Maarten Bosma, Gaurav Mishra, Adam Roberts,
Paul Barham, Hyung Won Chung, Charles Sutton,
Sebastian Gehrmann, et al. 2022. Palm: Scaling
language modeling with pathways. arXiv preprint
arXiv:2204.02311.
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian,
Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias
Plappert, Jerry Tworek, Jacob Hilton, Reiichiro
Nakano, et al. 2021. Training verifiers to solve math
word problems. arXiv preprint arXiv:2110.14168.
Gaole He, Yunshi Lan, Jing Jiang, Wayne Xin Zhao, and
Ji-Rong Wen. 2021. Improving multi-hop knowledge
base question answering by learning intermediate
supervision signals. In Proceedings of the 14th ACM
international conference on web search and data
mining, pages 553–561.
Dieuwke Hupkes, Mario Giulianelli, Verna Dankers,
Mikel Artetxe, Yanai Elazar, Tiago Pimentel, Chris-
tos Christodoulopoulos, Karim Lasri, Naomi Saphra,
9414
Arabella Sinclair, et al. 2022. State-of-the-art gen-
eralisation research in nlp: a taxonomy and review.
arXiv preprint arXiv:2210.03050.
Jinhao Jiang, Kun Zhou, Zican Dong, Keming Ye,
Wayne Xin Zhao, and Ji-Rong Wen. 2023. Structgpt:
A general framework for large language model to
reason on structured data.
Jinhao Jiang, Kun Zhou, Wayne Xin Zhao, and Ji-Rong
Wen. 2022. Unikgqa: Unified retrieval and reason-
ing for solving multi-hop question answering over
knowledge graph. arXiv preprint arXiv:2212.00959.
Jiho Kim, Sungjin Park, Yeonsu Kwon, Yohan Jo, James
Thorne, and Edward Choi. 2023. FactKG: Fact veri-
fication via reasoning on knowledge graphs. In Pro-
ceedings of the 61st Annual Meeting of the Associa-
tion for Computational Linguistics (Volume 1: Long
Papers), pages 16190–16206, Toronto, Canada. As-
sociation for Computational Linguistics.
Jens Lehmann, Robert Isele, Max Jakob, Anja Jentzsch,
Dimitris Kontokostas, Pablo N Mendes, Sebastian
Hellmann, Mohamed Morsey, Patrick Van Kleef,
Sören Auer, et al. 2015. Dbpedia–a large-scale, mul-
tilingual knowledge base extracted from wikipedia.
Semantic web, 6(2):167–195.
Shen-yun Miao, Chao-Chun Liang, and Keh-Yih Su.
2020. A diverse corpus for evaluating and developing
English math word problem solvers. In Proceedings
of the 58th Annual Meeting of the Association for
Computational Linguistics, pages 975–984, Online.
Association for Computational Linguistics.
OpenAI. 2023a. Gpt-4 technical report.
OpenAI. 2023b. Introducing chatgpt.
Arkil Patel, Satwik Bhattamishra, and Navin Goyal.
2021. Are NLP models really able to solve simple
math word problems? In Proceedings of the 2021
Conference of the North American Chapter of the
Association for Computational Linguistics: Human
Language Technologies, pages 2080–2094, Online.
Association for Computational Linguistics.
Apoorv Saxena, Aditay Tripathi, and Partha Talukdar.
2020. Improving multi-hop question answering over
knowledge graphs using knowledge base embeddings.
In Proceedings of the 58th Annual Meeting of the As-
sociation for Computational Linguistics, pages 4498–
4507, Online. Association for Computational Lin-
guistics.
Haitian Sun, Bhuwan Dhingra, Manzil Zaheer, Kathryn
Mazaitis, Ruslan Salakhutdinov, and William W Co-
hen. 2018. Open domain question answering using
early fusion of knowledge bases and text.
arXiv
preprint arXiv:1809.00782.
Alon Talmor, Jonathan Herzig, Nicholas Lourie, and
Jonathan Berant. 2019. CommonsenseQA: A ques-
tion answering challenge targeting commonsense
knowledge. In Proceedings of the 2019 Conference
of the North American Chapter of the Association for
Computational Linguistics: Human Language Tech-
nologies, Volume 1 (Long and Short Papers), pages
4149–4158, Minneapolis, Minnesota. Association for
Computational Linguistics.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten
Bosma, Ed Chi, Quoc Le, and Denny Zhou. 2022.
Chain of thought prompting elicits reasoning in large
language models. arXiv preprint arXiv:2201.11903.
Kun Xu, Yuxuan Lai, Yansong Feng, and Zhiguo Wang.
2019. Enhancing key-value memory neural networks
for knowledge based question answering. In Proceed-
ings of the 2019 Conference of the North American
Chapter of the Association for Computational Lin-
guistics: Human Language Technologies, Volume 1
(Long and Short Papers), pages 2937–2947, Min-
neapolis, Minnesota. Association for Computational
Linguistics.
Yuyu Zhang, Hanjun Dai, Zornitsa Kozareva, Alexan-
der J Smola, and Le Song. 2018. Variational reason-
ing for question answering with knowledge graph. In
AAAI.
Jie Zhou, Xu Han, Cheng Yang, Zhiyuan Liu, Lifeng
Wang, Changcheng Li, and Maosong Sun. 2019.
GEAR: Graph-based evidence aggregating and rea-
soning for fact verification. In Proceedings of the
57th Annual Meeting of the Association for Compu-
tational Linguistics, pages 892–901, Florence, Italy.
Association for Computational Linguistics.
9415
A
Prompts
The prompts for Sentence Segmentation, Graph
Retrieval, and Inference can be found in Table 4,
Table 5 and Table 6, respectively.
B
Relation Candidates Extraction
Algorithm
B.1
FACTKG
In FACTKG, we develop a new KG called Type-
DBpedia. This graph comprises types found in DB-
pedia and connects them through relations, thereby
enhancing the usability of KG content. We describe
the detailed process of incorporating Ri using DB-
pedia and TypeDBpedia in Algorithm 1.
We denote the entities as E1 and E2 because the
sub-sentence always includes two entities in FAC-
TKG. Relations (e, DBpedia) represents the set
of relations connected to e in DBpedia. Similarly,
Relations (T, TypeDBpedia) represents the set of
relations connected to T in TypeDBpedia.
Algorithm 1: Extract Relation Candidates
Input: Entity Set E = {E1, E2},
DBpedia, TypeDBpedia
Output: Relation Candidates Ri
Initialization: T = ∅, RT = ∅, RE = ∅,
Ri = ∅
for each entity e in E do
if isType (e) then
T ←T ∪{e}
end
else
if isEmpty (RE) then
RE ←Relations (e, DBpedia)
end
else
RE ←RE ∩Relations (e,
DBpedia)
end
end
end
if not isEmpty (T) then
RT ←Relations (T, TypeDBpedia)
Ri ←RE ∩RT
end
else
Ri ←RE
end
B.2
MetaQA
For the n-hop task in MetaQA, Ri is constructed
from the relations within n-hops from the seed
entity.
C
Experimental Settings
We utilize ChatGPT2 (OpenAI, 2023b) across all
tasks, and to acquire more consistent responses, we
carry out inference with the temperature and top_p
parameters set to 0.2 and 0.1, respectively. For each
stage of KG-GPT, 12 pieces of training samples
were made into in-context examples and added
to the prompt. In FACTKG, there are over 500
existing relations (|R| > 500), so we set k = 5 for
Top-K relation retrieval. Conversely, in MetaQA,
there are only 9 existing relations (|R| = 9), so we
set k = 3.
D
Data Examples
Examples of data from FACTKG and MetaQA can
be found in Tables 7 and 8, respectively.
E
Qualitative Results
E.1
FACTKG
Table 9 includes the graphs retrieved by KG-GPT,
along with the prediction results, for five different
claims.
E.2
MetaQA
Table 10 includes the graphs retrieved by KG-GPT,
along with the prediction results, for nine different
questions.
F
Top-K Relation Retrieval
F.1
FACTKG
The performance and the average number of re-
trieved triples are depicted in Table 11 and Table 12,
respectively.
F.2
MetaQA
The performance and the average number of re-
trieved triples are depicted in Table 13 and Table 14,
respectively.
2https://platform.openai.com/docs/guides/gpt/chat-
completions-api
9416
Sentence Segmentation Prompt
Please divide the given sentence into several sentences each of which can be represented by one triplet. The generated
sentences should be numbered and formatted as follows: #(number). (sentence), (entity set). The entity set for each
sentence should contain no more than two entities, with each entity being used only once in all statements. The ‘##’
symbol should be used to indicate an entity set. In the generated sentences, there cannot be more than two entities in
the entity set. (i.e., the number of ## must not be larger than two.)
Examples)
Sentence A: Ahmad Kadhim Assad’s club is Al-Zawra’a SC.
Entity set: [‘Ahmad_Kadhim_Assad’ ## “Al-Zawra’a_SC”]
–>Divided:
1. Ahmad Kadhim Assad’s club is Al-Zawra’a SC., Entity set: [‘Ahmad_Kadhim_Assad’ ## “Al-Zawra’a_SC”]
...
Sentence L: An academic journal with code IJPHDE is also Acta Math. Hungar.
Entity set: [“Acta Math. Hungar.” ## “IJPHDE”]
–>Divided:
1. An academic journal is with code IJPHDE., Entity set: [‘academic journal’ ## “IJPHDE”]
2. An academic journal is also Acta Math. Hungar., Entity set: [‘academic journal’ ## “Acta Math. Hungar.”]
Your Task)
Sentence: <<<<CLAIM>>>>
Entity set: <<<<ENTITY_SET>>>>
–>Divided:
Table 4: Sentence Segmentation Prompt.
Relation Retrieval Prompt
I will give you a set of words.
Find the top <<<<TOP_K>>>>elements from Words set which are most semantically related to the given sentence.
You may select up to <<<<TOP_K>>>>words. If there is nothing that looks semantically related, pick out any
<<<<TOP_K>>>>elements and give them to me.
Examples)
Sentence A: Ahmad Kadhim Assad’s club is Al-Zawra’a SC.
Words set: [‘club’, ‘clubs’, ‘parent’, ‘spouse’, ‘birthPlace’, ‘deathYear’, ‘leaderName’, ‘awards’, ‘award’,
‘vicepresident’, ‘vicePresident’]
Top 2 Answer: [‘club’, ‘clubs’]
...
Sentence L: An academic journal with code IJPHDE is also Acta Math. Hungar.
Words set: [‘abbreviation’, ‘placeOfBirth’, ‘owner’, ‘coden’, ‘almaMater’, ‘dean’, ‘coach’, ‘writer’, ‘firstAired’,
‘director’, ‘formerTeam’, ‘starring’, ‘birthPlace’]
Top 2 Answer: [‘abbreviation’, ‘coden’]
Now let’s find the top <<<<TOP_K>>>>elements.
Sentence: <<<<SENTENCE>>>>
Words set: <<<<RELATION_SET>>>>
Top <<<<TOP_K>>>>Answer:
Table 5: Relation Retrieval Prompt. The prompt is used when retrieving a relation to retrieve a graph.
9417
Inference Prompt
You should verify the claim based on the evidence set.
Each evidence is in the form of [head, relation, tail] and it means “head’s relation is tail.”.
Verify the claim based on the evidence set. (True means that everything contained in the claim is
supported by the evidence.)
Please note that the unit is not important. (e.g. “98400” is also same as 98.4kg)
Choose one of {True, False}, and give me the one-sentence evidence.
Examples)
Claim A: Ahmad Kadhim Assad’s club is Al-Zawra’a SC.
Evidence set: [[‘Ahamad_Kadhim’, ‘clubs’, “Al-Zawra’a SC”]]
Answer: True, based on the evidence set, Ahmad Kadhim Assad’s club is Al-Zawra’a SC.
...
Claim L: The place, designed by Huseyin Butuner and Hilmi Guner, is located in a country,
where the leader is Paul Nurse.
Evidence set: [[“Baku_Turkish_Martyrs’_Memorial”, ‘designer’, “Hüseyin Bütüner and Hilmi Güner”],
[“Baku_Turkish_Martyrs’_Memorial”, ‘location’, ‘Azerbaijan’]]
Answer: False, there is no evidence for Paul Nurse.
Now let’s verify the Claim based on the Evidence set.
Claim: <<<<CLAIM>>>>
Evidence set: <<<<EVIDENCE_SET>>>>
Answer:
Table 6: Inference Prompt.
Reasoning Type
Claim Example
Graph
One-hop
AIDAstella was built by Meyer Werft.
s
m
r2
Conjunction
AIDA Cruise line operated the AIDAstella which was built by
Meyer Werft.
c
s
m
r3
r2
Existence
Meyer Werft had a parent company.
m
r1
Multi-hop
AIDAstella was built by a company in Papenburg.
s
x
p
r2
r4
Negation
AIDAstella was not built by Meyer Werft in Papenburg.
s
m
p
r2
r4
Table 7: Five different reasoning types of FACTKG. r1: parentCompany, r2: shipBuilder, r3: shipOperator, r4:
location, m: Meyer Werft, s: AIDAstella, c: AIDA Cruises.
9418
Task
Question Examples
1-hop
1. what does [Helen Mack] star in?
2. what is the main language in [Karate-Robo Zaborgar]?
3. who is the writer of [Boyz n the Hood]?
2-hop
1. who are movie co-directors of [Delbert Mann]?
2. what genres do the films starred by [Al St. John] fall under?
3. which films share the screenwriter with [King Arthur]?
3-hop
1. the films that share directors with the film [Catch Me If You Can] were in which languages?
2. who are the directors of the movies written by the writer of [She]?
3. when did the movies release whose actors also appear in the movie [Operator 13]?
Table 8: Question examples from MetaQA.
Type
Claim
Retrieved Graph
Prediction
Conjunction
Yes, Agra Airport is located in India
where the leader is Narendra Modi.
[‘Agra_Airport’, location, ‘India’],
[‘India’, leader, ‘Narendra_Modi’],
[‘India’, leaderName, ‘Narendra_Modi’],
[‘Narendra_Modi’, birthPlace, ‘India’]
Supported
Conjunction
I wasn’t aware that 103 Colmore Row,
located in Birmingham, with 23
floors, was completed in 1976.
[‘103_Colmore_Row’, location, ‘Birmingham’],
[‘103_Colmore_Row’, floorCount, “23”],
[‘103_Colmore_Row’, completionDate, “1976”],
[‘103_Colmore_Row’, buildingEndDate, “1976”]
Supported
Multi-hop
Alfredo Zitarrosa died in a city,
Uruguay (which has Raul Fernando
Sendic Rodriguez as leader).
[‘Alfredo_Zitarrosa’, deathPlace, ‘Uruguay’],
[‘Alfredo_Zitarrosa’, birthPlace, ‘Uruguay’],
[‘Montevideo’, country, ‘Uruguay’],
[‘Alfredo_Zitarrosa’, deathPlace, ‘Montevideo’],
[‘Alfredo_Zitarrosa’, birthPlace, ‘Montevideo’],
[‘Uruguay’, capital, ‘Montevideo’],
[‘Uruguay’, leader, ‘Raúl_Fernando_Sendic_Rodríguez’],
[‘Uruguay’, leaderName, ‘Raúl_Fernando_Sendic_Rodríguez’]
Supported
Negation
Al-Taqaddum Air Base is located
in Fallujah which is not in Iraq.
[‘Al-Taqaddum_Air_Base’, city, ‘Fallujah’],
[‘Al-Taqaddum_Air_Base’, cityServed, ‘Fallujah’],
[‘Fallujah’, country, ‘Iraq’]
Refuted
Multi-hop
A country is the location of the
Adare Manor, is run by leader
Enda Kenny and the natives are
Irish people.
[‘Adare_Manor’, country, ‘Republic_of_Ireland’],
[‘Republic_of_Ireland’, leader, ‘Enda_Kenny’],
[‘Republic_of_Ireland’, leaderName, ‘Enda_Kenny’],
[‘Adare_Manor’, locationCountry, ‘Republic_of_Ireland’],
[‘Republic_of_Ireland’, demonym, ‘Irish_people’]
Supported
Table 9: Qualitative results from FACTKG.
Task
Question
Retrieved Graph
Prediction
what films does [Brigitte Nielsen] appear in?
[‘Cobra’, starred_actors, ‘Brigitte Nielsen’],
[‘Red Sonja’, starred_actors, ‘Brigitte Nielsen’]
‘Cobra’
1-hop
can you name a film directed by
[Nikolai Müllerschön]?
[‘The Red Baron’, directed_by, ‘Nikolai Müllerschön’]
‘The Red Baron’
what type of film is [Six Shooter]?
[‘Six Shooter’, has_genre, ‘Short’]
‘Short’
when did the films starred by
[Deborah Van Valkenburgh] release?
[‘Mean Guns’, starred_actors, ‘Deborah Van Valkenburgh’],
[‘Mean Guns’, release_year, ‘1997’]
‘1997’
2-hop
which films have the same director of
[The Duellists]?
[‘The Duellists’, directed_by, ‘Ridley Scott’],
[‘The Counselor’, directed_by, ‘Ridley Scott’]
‘The Counselor’
what genres are the movies written by
[Robert Kenner] in?
[‘Food, Inc.’, written_by, ‘Robert Kenner’],
[‘Food, Inc.’, has_genre, ‘Documentary’]
‘Documentary’
what are the genres of the movies whose
writers also wrote
[The Lives of a Bengal Lancer]?
[‘The Lives of a Bengal Lancer’, written_by, ‘John L. Balderston’],
[‘Frankenstein’, written_by, ‘John L. Balderston’],
[‘Frankenstein’, has_genre, ‘Horror’]
‘Horror’
3-hop
when did the movies starred by
[Seeking Justice] actors release?
[‘Seeking Justice’, starred_actors, ‘Nicolas Cage’],
[‘World Trade Center’, starred_actors, ‘Nicolas Cage’],
[‘World Trade Center’, release_year, ‘2006’]
‘2006’
what types are the movies starred by actors
in [A Thin Line Between Love and Hate]?
[‘A Thin Line Between Love and Hate’, directed_by, ‘Martin Lawrence’],
[“Big Momma’s House”, starred_actors, ‘Martin Lawrence’],
[“Big Momma’s House”, has_genre, ‘Comedy’],
[‘A Thin Line Between Love and Hate’, written_by, ‘Martin Lawrence’],
[‘A Thin Line Between Love and Hate’, starred_actors, ‘Martin Lawrence’]
‘Comedy’
Table 10: Qualitative results from MetaQA.
9419
Input Type
Method
k
Accuracy
With Evidence
KG-GPT
3
72.12
5
72.68
10
72.40
Table 11: Performance changes according to the change
in the k value in FACTKG.
k
FactKG
Supported
Refuted
3
1.93
1.08
5
2.52
1.39
10
3.13
1.86
Table 12: The average number of retrieved triples
changes according to the change in the k value in FAC-
TKG.
Method
k
MetaQA
1hop
MetaQA
2hop
MetaQA
3hop
1
97.0
93.4
89.4
KG-GPT
3
96.3
94.4
94.0
5
96.7
95.0
93.7
Table 13: Performance changes according to the change
in the k value in MetaQA.
k
MetaQA
1-hop
2-hop
3-hop
1
2.06
4.43
3.25
3
2.16
4.53
3.59
5
2.12
4.51
3.65
Table 14: The average number of retrieved triples
changes according to the change in the k value in
MetaQA.
9420
Motivation
Practical
Cognitive
Intrinsic
Fairness
4.1 4.2
Generalisation type
Compositional
Structural
Cross Task
Cross Language Cross Domain
Robustness
4.1 4.2
4.1 4.2
Shift type
Covariate
Label
Full
Assumed
4.1 4.2
Shift source
Naturally occuring
Partitioned natural
Generated shift
Fully generated
4.1 4.2
Shift locus
Train–test
Finetune train–test
Pretrain–train
Pretrain–test
4.1 4.2
Table 15: GenBench Evaluation Card from Hupkes et al. (2022).
9421


Question:
What is the primary loss function introduced by the SIU method?
A. Cross Entropy Loss
B. Token Masked Loss
C. Dual Masked KL-divergence Loss
D. Gradient Ascent Loss

Answer: C

Certain editing methods, which employ meta-learning or utilize classifiers (as shown in Table 1), necessitate the training of additional parameters or the implementation of extra network structures. Similar to Hyperparameters (Hparams), all Trainer classes inherit from a common base class, BaseTrainer. It includes essential attributes and abstract methods such as run and validate steps. Subclasses of the BaseTrainer define specific training steps for editing, such as calculating editing loss and locality loss, as well as the strategies for combining these losses. Once additional network structures are obtained, the subsequent editing process follows the same path as the Training-Free method. In EASYEDIT, various Trainers can be easily called with one click.

EasyEdit: An Easy-to-use Knowledge Editing Framework for
Large Language Models
Peng Wang♣, Ningyu Zhang♣∗, Bozhong Tian♣, Zekun Xi♣, Yunzhi Yao♣,
Ziwen Xu♣, Mengru Wang♣, Shengyu Mao♣, Xiaohan Wang♣, Siyuan Cheng♣,
Kangwei Liu♣, Yuansheng Ni♣, Guozhou Zheng♣, Huajun Chen♣∗,
♣Zhejiang University
 https://github.com/zjunlp/EasyEdit
Abstract
Large Language Models (LLMs) usually suffer
from knowledge cutoff or fallacy issues, which
means they are unaware of unseen events or
generate text with incorrect facts owing to out-
dated/noisy data. To this end, many knowledge
editing approaches for LLMs have emerged
– aiming to subtly inject/edit updated knowl-
edge or adjust undesired behavior while mini-
mizing the impact on unrelated inputs. Never-
theless, due to significant differences among
various knowledge editing methods and the
variations in task setups, there is no standard
implementation framework available for the
community, which hinders practitioners from
applying knowledge editing to applications. To
address these issues, we propose EASYEDIT,
an easy-to-use knowledge editing framework
for LLMs. It supports various cutting-edge
knowledge editing approaches and can be read-
ily applied to many well-known LLMs such as
T5, GPT-J, LlaMA, etc. Empirically, we report
the knowledge editing results on LlaMA-2 with
EASYEDIT, demonstrating that knowledge edit-
ing surpasses traditional fine-tuning in terms of
reliability and generalization. We have released
the source code on GitHub1, along with Google
Colab tutorials and comprehensive documenta-
tion2 for beginners to get started. Besides, we
present an online system3 for real-time knowl-
edge editing, and a demo video4.
1
Introduction
Large Language Models (LLMs) have revolution-
ized modern Natural Language Processing (NLP),
significantly improving performance across various
tasks (Brown et al., 2020; OpenAI, 2023; Anil et al.,
2023; Zhao et al., 2023; Touvron et al., 2023b;
∗Corresponding author.
1This is a subproject of KnowLM (https://github.
com/zjunlp/KnowLM), which facilitates knowledgeable LLM
Framework with EasyInstruct, EasyEdit, EasyDetect etc.
2https://zjunlp.gitbook.io/easyedit
3https://huggingface.co/spaces/zjunlp/EasyEdit
4https://youtu.be/Gm6T0QaaskU
Qiao et al., 2023; Zheng et al., 2023b; Pan et al.,
2023). However, deployed LLMs usually suffer
from knowledge cutoff or fallacy issues. For ex-
ample, LLMs such as ChatGPT and LlaMA pos-
sess information only up to their last training point.
They can sometimes produce inaccurate or mis-
leading information due to potential discrepancies
and biases in their pre-training data (Ji et al., 2023;
Hartvigsen et al., 2022). Hence, it’s essential to effi-
ciently update the parametric knowledge within the
LLMs to modify specific behaviors while avoiding
expensive retraining.
Indeed, finetuning or parameter-efficient finetun-
ing (Ding et al., 2022, 2023) offers methods for
modifying LLMs, these approaches can be com-
putationally expensive and may lead to overfitting,
particularly when applied to a limited number of
samples (Cao et al., 2021) or streaming errors of
LLMs. Additionally, fine-tuned models might for-
feit capabilities gained during pre-training, and
their modifications do not always generalize to rel-
evant inputs. An alternative methodology involves
using manually written or retrieved prompts to in-
fluence the LLMs’ output. These methods suffer
from reliability issues, as LLMs do not consistently
generate text aligned with the prefix prompt (Her-
nandez et al., 2023; Lewis et al., 2021). Addi-
tionally, due to the extensive amount of up-to-date
knowledge required for complex reasoning tasks,
the impracticality of context overload becomes in-
evitable whenever the context length is limited.
A feasible solution, knowledge editing5, aims to
efficiently modify the behavior of LLMs with mini-
mal impact on unrelated inputs. Research on knowl-
edge editing for LLMs (Meng et al., 2023, 2022;
Zheng et al., 2023a; Gupta et al., 2023; Mitchell
et al., 2022a; Geva et al., 2023; Hase et al., 2023;
Cohen et al., 2023a; Hartvigsen et al., 2023; Tan
et al., 2024; Yu et al., 2023) have displayed remark-
able progress across various tasks and settings.
5Knowledge editing can also be termed as model editing.
arXiv:2308.07269v3  [cs.CL]  24 Jun 2024
Editor
Evaluation
Reliability
apply_to_model
Method
User Demand
Hparams
LLMs
fθ
LLMs
fθe
model_main
Memory-based
Meta-Learning
Locate-Then-Edit
GRACE
ROME
MEND
…
…
…
Who is the president of the USA?
Joe Biden
xe =
ye =
model_hparams
Fluency
Locality
Portability
…
Editor: This class encapsulates the editor, which
can be single-instance, batch, sequential and multi-
modal editing, etc. according to user needs.
Hparams: This class manages the hyper-parameters
of various editing methods, such as the number of
modified layers, etc.
Evaluate: This class contains various metrics for eva-
luating editing performance(support customization)
Method: This class supports a variety of model editing
methods, including locating and modifying parameters,
meta-learning, etc.
Generalization
Figure 1: The overall architecture of EASYEDIT. The main function is apply_to_model, which applies the selected
editing method to the LLMs. The Editor serves as the direct entry point, receiving customized user inputs and
outputs, and returning the edited weights. Please note that some methods may require pre-training of classifiers or
hypernetworks through the Trainer (See §3.5). EASYEDIT supports customizable evaluation metrics.
However, these variations in both implementa-
tion and task settings have impeded the develop-
ment of a unified and comprehensive framework
for knowledge editing. Note that the complexity ob-
structs the direct comparison of effectiveness and
feasibility between different methods, and com-
plicates the creation of novel knowledge editing
approaches. To this end, we propose EASYEDIT,
an easy-to-use knowledge editing framework for
LLMs. EASYEDIT modularizes editing methods
and effectiveness evaluation while considering their
combination and interaction. It supports a vari-
ety of editing scenarios, including single-instance,
batch-instance, sequential, and multi-modal edit-
ing. Moreover, EASYEDIT provides evaluation
evaluations of key metrics such as Reliability, Gen-
eralization, Locality, and Portability (Yao et al.,
2023), to quantify the robustness and side effects
(Cohen et al., 2023b) of editing methods.
Specifically, in EASYEDIT, the Editor class in-
tegrates various editing components. The Method
class offers a unified interface apply_to_model,
which accepts editing descriptors and returns the
edited model, thereby facilitating the integration
of novel editing methodologies. Dedicated to eval-
uating editing performance, the Evaluate module
leverages metrics such as reliability, robust general-
ization, and locality. The Trainer module manages
the training of additional neural network structures.
Each module in EASYEDIT is meticulously defined,
striking a balance between cohesion and coupling.
Furthermore, we furnish examples of editing across
a spectrum of models, including T5 (Raffel et al.,
2019), GPT-J (Wang and Komatsuzaki, 2021), GPT-
NEO (Black et al., 2021), GPT2 (Radford et al.,
2019), LLaMA (Touvron et al., 2023a), LLaMA-2
(Touvron et al., 2023b), Mistral (Jiang et al., 2023),
and Qwen (Bai et al., 2023). We acknowledge
all the support for EASYEDIT, which is listed in
Appendix 6 due to space constraints.
2
Background
Previous Solutions
Despite the tremendous suc-
cess of LLMs in almost all NLP tasks, persis-
tent challenges such as knowledge cutoff and bi-
ased/toxic outputs remain. To counter these chal-
lenges, two approaches are generally employed:
1) FINE-TUNING: Traditional fine-tuning tech-
niques, along with delta tuning (Ding et al., 2022)
and LoRA tuning (Hu et al., 2021) utilize domain-
specific datasets to update the model’s internal para-
metric knowledge. However, these methods face
two notable challenges: First, they consume con-
siderable resources. Second, they risk the potential
of catastrophic forgetting (Ramasesh et al., 2022).
2) PROMPT-AUGMENTATION: Given a suffi-
cient number of demonstrations or retrieved con-
texts, LLMs can learn to enhance reasoning (Yu
et al., 2022) and generation through external knowl-
edge (Borgeaud et al., 2022; Guu et al., 2020;
Lewis et al., 2020). However, the performance
may be sensitive to factors such as the prompt-
ing template, the selection of in-context examples
(Zhao et al., 2021), or retrieved contexts (Ren et al.,
2023). These approaches also encounter the issue
of context length limitation (Liu et al., 2023a).
Knowledge Storage Mechanism
Within the
NLP literature, numerous studies have delved into
understanding the location of different types of
knowledge in language models (Petroni et al., 2019;
Roberts et al., 2020; Jiang et al., 2020). LLMs
can be conceptualized as knowledge banks, and
the transformer MLP layers function as key-value
memories according to observations from Geva
et al. (2021). This configuration promotes effi-
cient knowledge adjustments by precisely localiz-
ing knowledge within the MLP layers (denoted as
knowledge editing).
Knowledge editing enables nimble alterations
to the LLMs’ behavior through one data point.
Another promising attribute of knowledge edit-
ing is its ability to ensure the locality of edit-
ing, meaning that modifications are contained
within specific contexts. Additionally, the knowl-
edge editing technique can mitigate harmful lan-
guage generation (Geva et al., 2022). In this pa-
per, we present EASYEDIT, an easy-to-use knowl-
edge editing framework for LLMs. It seamlessly
integrates diverse editing technologies and sup-
ports the free combination of modules for various
LLMs. Through its unified framework and inter-
face, EASYEDIT enables users to swiftly compre-
hend and apply the prevalent knowledge editing
methods included in the package.
3
Design and Implementation
EASYEDIT provides a complete editing and evalu-
ation process built on Pytorch (Paszke et al., 2019)
and Huggingface (Wolf et al., 2020). This section
commences with an exploration of the assemblabil-
ity aspect of EASYEDIT, followed by a detailed ex-
planation of the design and implementation of each
component within the EASYEDIT framework (as
shown in Figure 1). Additionally, we demonstrate
a straightforward example of applying MEND to
LLaMA, altering the output of the U.S. President
to Joe Biden.
3.1
Assemblability
In the realm of knowledge editing, various dis-
tinct scenarios6 exist. To cater to this diversity,
EASYEDIT offers flexible combinations of mod-
ules that different editing Editor (such as single-
instance, batch-instance (details in Appendix A)),
6Denoted as (Editor, METHOD, TARGET)
Figure 2: A running example of knowledge editing for
LLMs in EASYEDIT. Utilizing the MEND approach,
we can successfully transform the depiction of the U.S.
President into that of Joe Biden.
METHOD (such as ROME, GRACE (§3.3)). About
editing TARGET, EASYEDIT can accommodate any
parameterized white-box existing model. Addition-
ally, recent research (Dong et al., 2022) indicates
that LLMs exhibit robust in-context learning capa-
bilities. By providing edited facts to LLMs, one
can alter the behavior of black-box models such
as GPT4 (OpenAI, 2023). All those combinations
are easily implementable and verifiable within the
EASYEDIT framework.
3.2
Editor
The Editor serves a pivotal role in knowledge
editing as it directly establishes the editing tasks
and corresponding editing scenarios. Users sup-
ply the editor descriptor (xe) and the edit target
(ye), but the input format varies according to the
different editing objects. For instance, in Seq2Seq
models, the edit target typically serves as the de-
coder’s input, while in autoregressive models, xe
and ye need to be concatenated to maximize the
conditional probability. To facilitate unified edit-
ing across diverse architecture models, we metic-
ulously develop a component prepare_requests
to transform editing inputs.
In EASYEDIT, we provide an “edit” interface, in-
corporating components such as Hparams, Method,
and Evaluate. During the editing phase, various
knowledge editing strategies can be executed by in-
voking the apply_to_model function available in
all different methods, it also performs evaluations
Method
Batch
Edit
Sequential
Edit
Additional
Train
Edit
Area
Time (s)
VRAM (GB)
Memory-based
SERAC
YES
YES
YES
External Model
8.46
42
IKE
NO
NO
YES
In-Context
4.57
52
GRACE
NO
YES
NO
MLP+codebook
142.68
28
MELO
YES
YES
NO
LoRA+codebook
154.32
30
Meta-learning
KE
YES
YES
YES
MLP
7.87
49
MEND
YES
YES
YES
MLP
6.39
46
Locate-Then-Edit
KN
NO
YES
NO
MLP
425.64
42
ROME
NO
YES
NO
MLP
187.90
31
MEMIT
YES
YES
NO
MLP
169.28
33
PMET
YES
YES
NO
MLP
219.17
34
Table 1: Comparison of several model editing methods. ‘Batch Edit’ refers to simultaneously editing multiple target
knowledge instances. ‘Sequential Edit’ refers to maintaining previously edited knowledge while performing new
edits. ‘Additional Train’ refers to the need for pre-training other network structures or parameters before editing.
‘Edit Area’ indicates the location of the edit, with MLP representing the linear layer. ‘Time & VRAM’ reflects the
efficiency of the editing method (using LlaMA-7B as an example). ‘Time’ indicates the wall clock time required for
conducting 10 edits, while VRAM represents the graphics memory usage.
of the model before and after the editing to gauge
the editing’s multifaceted impact on the model be-
havior, including generalization and side effects.
An example to edit through EASYEDIT is depicted
in Figure 2.
Note that the ability to execute batch editing
(multiple edits in a single instance) and sequential
editing (implementing new edits while preserving
previous editing) is a crucial feature of knowledge
editing (Huang et al., 2023). For methods that
support batch editing, editing instances are inputted
in chunk form. In addition, EASYEDIT provides
a boolean switch, enabling users to either retain
the pre-edit weights for single-instance editing or
discard them for sequential editing.
3.3
Method
As the core component of knowledge editing, edit-
ing methods alter the model’s behavior by modi-
fying its internal parameters (e.g. MLP, Attention
Mechanisms) or explicitly utilizing preceding edit-
ing facts, among other strategies. Impressive re-
lated works (Table 1) abound in this field, and they
can be generally grouped into three categories as
proposed by Yao et al. (2023).
Memory-based
This category, encompassing
methods such as SERAC (Mitchell et al., 2022b),
IKE (Zheng et al., 2023a), and GRACE (Hartvigsen
et al., 2023), emphasizes the use of memory ele-
ments to store and manipulate information during
editing. SERAC applies retrieval and classification
routing, GRACE replaces hidden states with pa-
rameters searched from a codebook for edit memo-
rization, while IKE uses context-edit facts to guide
the model in generating edited facts.
Meta-learning
These methods learn the weight
updates (denoted as ∆), which are then added to
the original weights for editing. Examples include
KE (Cao et al., 2021), which uses a bidirectional-
LSTM to predict weight updates, and MEND
(Mitchell et al., 2022a), which adjusts model param-
eters through low-rank decomposition of gradients.
Locate-Then-Edit
This paradigm focuses on
knowledge localization to modify the parameters
of specific neurons responsible for storing the edit-
ing facts. EASYEDIT integrates methods like KN
(Dai et al., 2021), which employs gradient-based
methods to update specific neurons. Moreover,
EASYEDIT supports ROME (Meng et al., 2023),
PMET (Li et al., 2024) and MEMIT (Meng et al.,
2022), leveraging causal intervention to pinpoint
knowledge within a specific MLP layer and en-
abling the modification of the entire matrix.
However, it is not practical to expose the editing
methods directly to users due to the complexity of
the underlying concepts and the time investment
required to understand them. Additionally, dif-
ferences in input-output formats across methods
could further complicate the learning process. To
circumvent these hurdles, we implement a unified
interface, apply_to_model, in EASYEDIT. Align-
ing with the Strategy design pattern, this interface
is designed to be overridden by different types of
editing methods, ensuring consistent input and out-
In-Scope
 Who is the president
of the USA?
xe :
 Who holds the position of
the president in the USA?
x′￼e :
 The America's president is who？
x′￼e :
 Who is the wife of the
USA President?
x′￼′￼
e :
What is the capital of America?
who is the actress that plays
rose in Titanic
Out-of-Scope
Reliability
Portability
Locality
Generalization
Same/Other Distribution
Figure 3: Depiction of the edit scope for edit descriptor
Who is the president of the USA? It contains an example
for knowledge editing evaluation, including Reliability,
Generalization, Portability, and Locality.
put types. Specifically, it accepts a ‘request’ that
includes the editing descriptor, the target of the edit,
and any input data necessary to evaluate the editing
performance. After processing the request(s), the
interface returns the edited model weights. This
design ensures both flexibility and easy-to-use, en-
abling users to handle knowledge editing instances
effortlessly and utilize the customized models in
other downstream tasks.
3.4
Hparams
When initializing an editing method, it is crucial
to specify the related hyperparameters. These in-
clude the model to be edited, the layers targeted
for modification, and, optionally, the type of ex-
ternal model, among other parameters. For meth-
ods that alter the LLMs’ internal parameters, the
adjustable parameter names should be indicated
using the MODULE_NAME format, such as trans-
former.h.5.mlp.fc_out. In this case, the parameters
of the fc_out linear layer in the fifth layer MLP
of GPT-J would be modified, while all other pa-
rameters remain frozen. Layer selection adheres
to the locality of knowledge (Meng et al., 2023)
or retains layers with higher success rates in pilot
experiments (Mitchell et al., 2022a), as elaborated
in Appendix B.
All hyperparameter classes derive from a com-
mon base class, Hyperparams, which includes
necessary attributes and abstract methods. This
base class supports loading hyperparameters in
both yaml and json formats.
Moreover, the
Hyperparams base class can be used to initialize
the Trainer module, streamlining the workflow.
3.5
Trainer
Certain editing methods, which employ meta-
learning or utilize classifiers (as shown in Table
1), necessitate the training of additional parame-
ters or the implementation of extra network struc-
tures. Similar to Hyperparameters (Hparams), all
Trainer classes inherit from a common base class,
BaseTrainer. It includes essential attributes and
abstract methods such as run and validate steps.
Subclasses of the BaseTrainer define specific
training steps for editing, such as calculating edit-
ing loss and locality loss, as well as the strategies
for combining these losses. Once additional net-
work structures are obtained, the subsequent edit-
ing process follows the same path as the Training-
Free method. In EASYEDIT, various Trainers can
be easily called with one click.
4
Evaluation
Knowledge editing, as defined by Mitchell et al.
(2022b), involves supplying a specific editing de-
scriptor xe (input instance) and an editing target ye
(desired output). From these, an editing instance
ze is generated in the form: ze ∼[xe, ye]. The
goal is to adjust the behavior of the initial base
model fθ (where θ represents the model’s param-
eters) to produce an edited model fθe. Ideally, for
the editing instance, the edited model would be-
have such that fθe(xe) = ye. Additionally, the
editing scope S(ze) refers to a set of input exam-
ples whose true labels have been influenced by the
editing instance. In most cases, a successful edit
should affect the model’s predictions for numerous
In-Scope (I(xe) ∼{x′
e|x′
e ∈S(ze)}) inputs, while
leaving Out-of-Scope (O(xe) ∼{x′
e|x′
e /∈S(ze)})
inputs unchanged.
We employ six dimensions of metrics to assess
the performance of editing methods, including Re-
liability, Generalization, Locality, Portability,
Fluency (Zhang et al., 2018) and Efficiency (as
shown in Figure 3).
Reliability
This metric measures the average ac-
curacy on the given editing instance ze.
Generalization
The edit should appropriately in-
fluence in-scope inputs, this metric gauges the av-
erage accuracy on in-scope inputs I(xe).
Locality
Editing should adhere to the principle
of locality, it evaluates whether out-of-scope inputs
O(xe) can remain unchanged as the base model.
Portability
The robust generalization of the edit,
assessing whether the edited knowledge can be
effectively applied to related content.
Fluency
It measures the weighted average of bi-
gram and tri-gram entropies to assess the diversity
of text generations.
Efficiency
Editing should be time and resource-
efficient. This metric quantifies efficiency by mea-
suring editing time and VRAM consumption.
5
Experiments
In this section, we will outline the experiment set-
ting and report the empirical results of multiple
editing methods supported in EASYEDIT (Table 2).
5.1
Experiment Setting
To validate the potential application of knowledge
editing on LLMs, we utilize LlaMA 2 (7B) (Tou-
vron et al., 2023b), a model with a large parameter
size, representing the decoder-only structure.
We employ the ZsRE dataset to test the capabil-
ity of knowledge editing in incorporating substan-
tial and general fact associations into the model.
ZsRE (Levy et al., 2017) is a question-answering
(QA) dataset that generates an equivalence neigh-
bor through back-translation. Later, it is further
expanded by Yao et al. (2023) to provide a more
comprehensive evaluation of knowledge editing,
including an assessment of the LLMs’ ability to
integrate the edited fact with other facts related
to the target object o* (an aspect of Portability).
For baselines, we compare various editing methods
and additionally employ FT-L from ROME (Meng
et al., 2023). FT-L updates parameters for a single
MLP layer and applies an L∞norm constraint to
limit the weight changes.
5.2
Experiment Results
Table 2 reveals SERAC and IKE’s superior perfor-
mance on the ZsRE datasets, exceeding 99% on
several metrics. While ROME and MEMIT per-
form sub-optimally in generalization, they exhibit
relatively high performance in terms of reliability
and locality. IKE exhibits the potential of gradient-
free updates through in-context learning, leading
to near-perfect scores in both reliability and gen-
eralization. However, it shows some deficiency
in locality, as preceding prompts may influence
out-of-scope inputs. GRACE exhibits poor gener-
alization, possibly attributed to the lack of explicit
semantic representation in its activations within
Reliability
Generalization
Locality
Portability
Fluency
FT-L
56.94
52.02
96.32
51.03
488.41
SERAC
99.49
99.13
100.00
57.82
423.22
IKE
100.00
99.98
69.19
67.56
557.37
MEND
94.24
90.27
97.04
56.95
540.06
KN
28.95
28.43
65.43
37.18
478.32
ROME
92.45
87.04
99.63
57.47
587.58
MEMIT
92.94
85.97
99.49
60.64
576.51
GRACE
99.22
0.43
100.00
56.87
426.31
Table 2: Editing results of the four metrics on LlaMA-2
using EASYEDIT. The settings for the model and the
dataset are the same with Yao et al. (2023).
the decoder-only model (Liu et al., 2023b). FT-L’s
performance on ZsRE falls significantly short com-
pared to ROME, even though both methods modify
the same layer parameters. This suggests that under
the norm constraint, fine-tuning is not an effective
strategy for knowledge editing. MEND performs
well overall, achieving over 90% accuracy on multi-
ple metrics and even surpassing ROME in terms of
reliability and generalization. KN performs poorly,
indicating that it may be better suited for editing
tasks in smaller models or tasks involving knowl-
edge attribution.
For the Portability evaluation, where the infer-
ence depends on a single connection or ‘hop’ be-
tween facts, most editing methods struggle to ef-
fectively combine the edited fact with other facts
relevant to the target object o*. While SERAC
obtains good performance on previous metrics, it
completely fails to propagate the edited knowledge.
This is because SERAC utilizes an external model
with a smaller parameter size for counterfactual
routing whereas the smaller model struggles to re-
call a rich set of relevant facts. IKE still maintains a
relatively high capability for ripple editing (exceed-
ing 67%), demonstrating that in-context learning is
a promising approach to propagate edited knowl-
edge to other related facts.
6
Conclusion and Future work
We propose EASYEDIT, an easy-to-use knowledge
editing framework for LLMs, which supports many
cutting-edge approaches and various LLMs. The
ability to edit and manipulate LLMs in a controlled
and targeted manner may open up new possibili-
ties for knowledge augmentation (Wu et al., 2023,
2020; Zhang et al., 2022; Chen et al., 2022) and
adaptation across various natural language process-
ing tasks (Kaddour et al., 2023). In the future, we
will continue to integrate advanced editing tech-
nologies into EASYEDIT, aiming at facilitating fur-
ther research and inspiring new ideas for the NLP
community.
Acknowledgments
We thank the developers of the ROME7 library for
their significant contributions to the NLP commu-
nity. We are grateful to Ting Lu and Yu Zhang
who participated in the development of this project
during the Zhejiang University Summer Camp. We
also extend our gratitude to the NLP team at East
China Normal University, particularly Lang Yu, for
their support of the Melo module. Special thanks
to Tom Hartvigsen for his contributions to the im-
plementation of GRACE. We are grateful to the
TMG-NUDT team for their valuable suggestions
and technical support for the PMET method. We
are grateful to Jia-Chen Gu from the University of
California, Los Angeles, and Haiyang Yu from the
Department of Cyberspace Security, University of
Science for their constructive suggestions on de-
velopment of EASYEDIT. We thank Yiquan Wu
and Zeqing Yuan for helping the AAAI 2024 tuto-
rial (canceled since part of speakers cannot present
in person) of EasyEdit. Appreciation is also ex-
tended to all PR contributors, and issue feedback
providers during the EasyEdit version iterations, es-
pecially Damien de Mijolla for proposing different
optimization goals for FT, which complemented
the fine-tuning baseline, and to Yuxuan Zhai for
pointing out the portability metric evaluation issue
of LlaMA-2-7B.
We would like to express gratitude to the
anonymous reviewers for their kind comments.
This work was supported by the National Natu-
ral Science Foundation of China (No. 62206246,
No.
NSFCU23B2055, No.
NSFCU19B2027),
the Fundamental Research Funds for the Central
Universities (226-2023-00138), Zhejiang Provin-
cial Natural Science Foundation of China (No.
LGG22F030011), Yongjiang Talent Introduction
Programme (2021A-156-G), CCF-Tencent Rhino-
Bird Open Research Fund, Tencent AI Lab Rhino-
Bird Focused Research Program (RBFR2024003),
Information Technology Center and State Key Lab
of CAD&CG, Zhejiang University.
Ethics Statement
The significance of knowledge editing lies in its
direct impact on the behavior and output results of
LMs. Malicious edits may lead to the generation
of responses with toxicity or bias in LMs, posing
potential harm to users and society. Therefore,
7https://github.com/kmeng01/rome
when applying knowledge editing techniques or
utilizing this system, careful consideration must be
given to potential risks and ethical concerns. All
our data undergoes meticulous manual inspection,
and any malicious edits or offensive content have
been removed.
References
Rohan Anil, Andrew M. Dai, Orhan Firat, Melvin John-
son, Dmitry Lepikhin, Alexandre Passos, Siamak
Shakeri, Emanuel Taropa, Paige Bailey, Zhifeng
Chen, Eric Chu, Jonathan H. Clark, Laurent El
Shafey, Yanping Huang, Kathy Meier-Hellstern, Gau-
rav Mishra, Erica Moreira, Mark Omernick, Kevin
Robinson, Sebastian Ruder, Yi Tay, Kefan Xiao,
Yuanzhong Xu, Yujing Zhang, Gustavo Hernandez
Abrego, Junwhan Ahn, Jacob Austin, Paul Barham,
Jan Botha, James Bradbury, Siddhartha Brahma,
Kevin Brooks, Michele Catasta, Yong Cheng, Colin
Cherry, Christopher A. Choquette-Choo, Aakanksha
Chowdhery, Clément Crepy, Shachi Dave, Mostafa
Dehghani, Sunipa Dev, Jacob Devlin, Mark Díaz,
Nan Du, Ethan Dyer, Vlad Feinberg, Fangxiaoyu
Feng, Vlad Fienber, Markus Freitag, Xavier Gar-
cia, Sebastian Gehrmann, Lucas Gonzalez, Guy Gur-
Ari, Steven Hand, Hadi Hashemi, Le Hou, Joshua
Howland, Andrea Hu, Jeffrey Hui, Jeremy Hur-
witz, Michael Isard, Abe Ittycheriah, Matthew Jagiel-
ski, Wenhao Jia, Kathleen Kenealy, Maxim Krikun,
Sneha Kudugunta, Chang Lan, Katherine Lee, Ben-
jamin Lee, Eric Li, Music Li, Wei Li, YaGuang Li,
Jian Li, Hyeontaek Lim, Hanzhao Lin, Zhongtao Liu,
Frederick Liu, Marcello Maggioni, Aroma Mahendru,
Joshua Maynez, Vedant Misra, Maysam Moussalem,
Zachary Nado, John Nham, Eric Ni, Andrew Nys-
trom, Alicia Parrish, Marie Pellat, Martin Polacek,
Alex Polozov, Reiner Pope, Siyuan Qiao, Emily Reif,
Bryan Richter, Parker Riley, Alex Castro Ros, Au-
rko Roy, Brennan Saeta, Rajkumar Samuel, Renee
Shelby, Ambrose Slone, Daniel Smilkov, David R.
So, Daniel Sohn, Simon Tokumine, Dasha Valter,
Vijay Vasudevan, Kiran Vodrahalli, Xuezhi Wang,
Pidong Wang, Zirui Wang, Tao Wang, John Wiet-
ing, Yuhuai Wu, Kelvin Xu, Yunhan Xu, Linting
Xue, Pengcheng Yin, Jiahui Yu, Qiao Zhang, Steven
Zheng, Ce Zheng, Weikang Zhou, Denny Zhou, Slav
Petrov, and Yonghui Wu. 2023. Palm 2 technical
report.
Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang,
Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei
Huang, Binyuan Hui, Luo Ji, Mei Li, Junyang Lin,
Runji Lin, Dayiheng Liu, Gao Liu, Chengqiang Lu,
Keming Lu, Jianxin Ma, Rui Men, Xingzhang Ren,
Xuancheng Ren, Chuanqi Tan, Sinan Tan, Jianhong
Tu, Peng Wang, Shijie Wang, Wei Wang, Sheng-
guang Wu, Benfeng Xu, Jin Xu, An Yang, Hao Yang,
Jian Yang, Shusheng Yang, Yang Yao, Bowen Yu,
Hongyi Yuan, Zheng Yuan, Jianwei Zhang, Xingx-
uan Zhang, Yichang Zhang, Zhenru Zhang, Chang
Zhou, Jingren Zhou, Xiaohuan Zhou, and Tianhang
Zhu. 2023. Qwen technical report.
Sid Black, Leo Gao, Phil Wang, Connor Leahy,
and Stella Biderman. 2021.
GPT-Neo:
Large
Scale Autoregressive Language Modeling with Mesh-
Tensorflow. If you use this software, please cite it
using these metadata.
Sebastian Borgeaud, Arthur Mensch, Jordan Hoff-
mann, Trevor Cai, Eliza Rutherford, Katie Milli-
can, George Bm Van Den Driessche, Jean-Baptiste
Lespiau, Bogdan Damoc, Aidan Clark, et al. 2022.
Improving language models by retrieving from tril-
lions of tokens. In International conference on ma-
chine learning, pages 2206–2240. PMLR.
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie
Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind
Neelakantan, Pranav Shyam, Girish Sastry, Amanda
Askell, Sandhini Agarwal, Ariel Herbert-Voss,
Gretchen Krueger, Tom Henighan, Rewon Child,
Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu,
Clemens Winter, Christopher Hesse, Mark Chen, Eric
Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess,
Jack Clark, Christopher Berner, Sam McCandlish,
Alec Radford, Ilya Sutskever, and Dario Amodei.
2020. Language models are few-shot learners.
Nicola De Cao, Wilker Aziz, and Ivan Titov. 2021. Edit-
ing factual knowledge in language models.
Xiang Chen, Ningyu Zhang, Xin Xie, Shumin Deng,
Yunzhi Yao, Chuanqi Tan, Fei Huang, Luo Si, and
Huajun Chen. 2022.
KnowPrompt: Knowledge-
aware prompt-tuning with synergistic optimization
for relation extraction. In Proceedings of the ACM
Web Conference 2022. ACM.
Roi Cohen, Eden Biran, Ori Yoran, Amir Globerson,
and Mor Geva. 2023a. Evaluating the ripple effects
of knowledge editing in language models. CoRR,
abs/2307.12976.
Roi Cohen, Eden Biran, Ori Yoran, Amir Globerson,
and Mor Geva. 2023b. Evaluating the ripple effects
of knowledge editing in language models.
Damai Dai, Li Dong, Yaru Hao, Zhifang Sui, and Furu
Wei. 2021. Knowledge neurons in pretrained trans-
formers. CoRR, abs/2104.08696.
Ning Ding, Yujia Qin, Guang Yang, Fuchao Wei, Zong-
han Yang, Yusheng Su, Shengding Hu, Yulin Chen,
Chi-Min Chan, Weize Chen, Jing Yi, Weilin Zhao,
Xiaozhi Wang, Zhiyuan Liu, Hai-Tao Zheng, Jianfei
Chen, Yang Liu, Jie Tang, Juanzi Li, and Maosong
Sun. 2022. Delta tuning: A comprehensive study of
parameter efficient methods for pre-trained language
models.
Ning Ding, Yujia Qin, Guang Yang, Fuchao Wei,
Zonghan Yang, Yusheng Su, Shengding Hu, Yulin
Chen, Chi-Min Chan, Weize Chen, et al. 2023.
Parameter-efficient fine-tuning of large-scale pre-
trained language models. Nature Machine Intelli-
gence, 5(3):220–235.
Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Zhiy-
ong Wu, Baobao Chang, Xu Sun, Jingjing Xu, and
Zhifang Sui. 2022. A survey for in-context learning.
arXiv preprint arXiv:2301.00234.
Mor Geva, Jasmijn Bastings, Katja Filippova, and Amir
Globerson. 2023. Dissecting recall of factual asso-
ciations in auto-regressive language models. CoRR,
abs/2304.14767.
Mor Geva, Avi Caciularu, Kevin Wang, and Yoav Gold-
berg. 2022. Transformer feed-forward layers build
predictions by promoting concepts in the vocabulary
space. In Proceedings of the 2022 Conference on
Empirical Methods in Natural Language Process-
ing, pages 30–45, Abu Dhabi, United Arab Emirates.
Association for Computational Linguistics.
Mor Geva, Roei Schuster, Jonathan Berant, and Omer
Levy. 2021. Transformer feed-forward layers are key-
value memories. In Proceedings of the 2021 Confer-
ence on Empirical Methods in Natural Language Pro-
cessing, pages 5484–5495, Online and Punta Cana,
Dominican Republic. Association for Computational
Linguistics.
Anshita Gupta, Debanjan Mondal, Akshay Krishna
Sheshadri, Wenlong Zhao, Xiang Lorraine Li,
Sarah Wiegreffe, and Niket Tandon. 2023.
Edit-
ing commonsense knowledge in GPT.
CoRR,
abs/2305.14956.
Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasu-
pat, and Mingwei Chang. 2020. Retrieval augmented
language model pre-training. In International confer-
ence on machine learning, pages 3929–3938. PMLR.
Thomas Hartvigsen, Saadia Gabriel, Hamid Palangi,
Maarten Sap, Dipankar Ray, and Ece Kamar. 2022.
ToxiGen: A large-scale machine-generated dataset
for adversarial and implicit hate speech detection.
In Proceedings of the 60th Annual Meeting of the
Association for Computational Linguistics (Volume
1: Long Papers), pages 3309–3326, Dublin, Ireland.
Association for Computational Linguistics.
Thomas Hartvigsen, Swami Sankaranarayanan, Hamid
Palangi, Yoon Kim, and Marzyeh Ghassemi. 2023.
Aging with grace: Lifelong model editing with dis-
crete key-value adaptors.
Peter Hase, Mohit Bansal, Been Kim, and Asma Ghan-
deharioun. 2023. Does localization inform editing?
surprising differences in causality-based localization
vs. knowledge editing in language models. CoRR,
abs/2301.04213.
Evan Hernandez, Belinda Z. Li, and Jacob Andreas.
2023. Inspecting and editing knowledge representa-
tions in language models.
Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan
Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and
Weizhu Chen. 2021. Lora: Low-rank adaptation of
large language models.
Zeyu Huang, Yikang Shen, Xiaofeng Zhang, Jie Zhou,
Wenge Rong, and Zhang Xiong. 2023. Transformer-
patcher: One mistake worth one neuron.
Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan
Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea
Madotto, and Pascale Fung. 2023. Survey of halluci-
nation in natural language generation. ACM Comput-
ing Surveys, 55(12):1–38.
Albert Q. Jiang, Alexandre Sablayrolles, Arthur Men-
sch, Chris Bamford, Devendra Singh Chaplot, Diego
de las Casas, Florian Bressand, Gianna Lengyel, Guil-
laume Lample, Lucile Saulnier, Lélio Renard Lavaud,
Marie-Anne Lachaux, Pierre Stock, Teven Le Scao,
Thibaut Lavril, Thomas Wang, Timothée Lacroix,
and William El Sayed. 2023. Mistral 7b.
Zhengbao Jiang, Frank F. Xu, Jun Araki, and Graham
Neubig. 2020. How can we know what language
models know? Transactions of the Association for
Computational Linguistics, 8:423–438.
Jean Kaddour, Joshua Harris, Maximilian Mozes, Her-
bie Bradley, Roberta Raileanu, and Robert McHardy.
2023. Challenges and applications of large language
models. CoRR, abs/2307.10169.
Yoonho Lee, Annie S. Chen, Fahim Tajwar, Ananya
Kumar, Huaxiu Yao, Percy Liang, and Chelsea Finn.
2023. Surgical fine-tuning improves adaptation to
distribution shifts.
Omer Levy, Minjoon Seo, Eunsol Choi, and Luke
Zettlemoyer. 2017. Zero-shot relation extraction via
reading comprehension. In Proceedings of the 21st
Conference on Computational Natural Language
Learning (CoNLL 2017), pages 333–342, Vancouver,
Canada. Association for Computational Linguistics.
Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio
Petroni, Vladimir Karpukhin, Naman Goyal, Hein-
rich Küttler, Mike Lewis, Wen-tau Yih, Tim Rock-
täschel, et al. 2020. Retrieval-augmented generation
for knowledge-intensive nlp tasks. Advances in Neu-
ral Information Processing Systems, 33:9459–9474.
Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio
Petroni, Vladimir Karpukhin, Naman Goyal, Hein-
rich Küttler, Mike Lewis, Wen tau Yih, Tim Rock-
täschel, Sebastian Riedel, and Douwe Kiela. 2021.
Retrieval-augmented generation for knowledge-
intensive nlp tasks.
Xiaopeng Li, Shasha Li, Shezheng Song, Jing Yang, Jun
Ma, and Jie Yu. 2024. Pmet: Precise model editing
in a transformer.
Nelson F Liu, Kevin Lin, John Hewitt, Ashwin Paran-
jape, Michele Bevilacqua, Fabio Petroni, and Percy
Liang. 2023a.
Lost in the middle:
How lan-
guage models use long contexts.
arXiv preprint
arXiv:2307.03172.
Tian Yu Liu, Matthew Trager, Alessandro Achille, Pra-
muditha Perera, Luca Zancato, and Stefano Soatto.
2023b. Meaning representations from trajectories in
autoregressive models.
Kevin Meng, David Bau, Alex Andonian, and Yonatan
Belinkov. 2023. Locating and editing factual associa-
tions in gpt.
Kevin Meng, Arnab Sen Sharma, Alex Andonian,
Yonatan Belinkov, and David Bau. 2022.
Mass-
editing memory in a transformer.
Eric Mitchell, Charles Lin, Antoine Bosselut, Chelsea
Finn, and Christopher D. Manning. 2022a.
Fast
model editing at scale.
Eric Mitchell, Charles Lin, Antoine Bosselut, Christo-
pher D. Manning, and Chelsea Finn. 2022b. Memory-
based model editing at scale.
OpenAI. 2023. Gpt-4 technical report.
Shirui Pan, Linhao Luo, Yufei Wang, Chen Chen, Ji-
apu Wang, and Xindong Wu. 2023. Unifying large
language models and knowledge graphs: A roadmap.
CoRR, abs/2306.08302.
Adam Paszke, Sam Gross, Francisco Massa, Adam
Lerer, James Bradbury, Gregory Chanan, Trevor
Killeen, Zeming Lin, Natalia Gimelshein, Luca
Antiga, Alban Desmaison, Andreas Köpf, Edward
Yang, Zach DeVito, Martin Raison, Alykhan Tejani,
Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Jun-
jie Bai, and Soumith Chintala. 2019. Pytorch: An
imperative style, high-performance deep learning li-
brary.
Fabio Petroni, Tim Rocktäschel, Sebastian Riedel,
Patrick Lewis, Anton Bakhtin, Yuxiang Wu, and
Alexander Miller. 2019. Language models as knowl-
edge bases?
In Proceedings of the 2019 Confer-
ence on Empirical Methods in Natural Language Pro-
cessing and the 9th International Joint Conference
on Natural Language Processing (EMNLP-IJCNLP),
pages 2463–2473, Hong Kong, China. Association
for Computational Linguistics.
Shuofei Qiao, Yixin Ou, Ningyu Zhang, Xiang Chen,
Yunzhi Yao, Shumin Deng, Chuanqi Tan, Fei Huang,
and Huajun Chen. 2023. Reasoning with language
model prompting: A survey. In Proceedings of the
61st Annual Meeting of the Association for Compu-
tational Linguistics (Volume 1: Long Papers), pages
5368–5393, Toronto, Canada. Association for Com-
putational Linguistics.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan,
Dario Amodei, Ilya Sutskever, et al. 2019. Language
models are unsupervised multitask learners. OpenAI
blog, 1(8):9.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine
Lee, Sharan Narang, Michael Matena, Yanqi Zhou,
Wei Li, and Peter J. Liu. 2019. Exploring the limits
of transfer learning with a unified text-to-text trans-
former. CoRR, abs/1910.10683.
Vinay Venkatesh Ramasesh, Aitor Lewkowycz, and
Ethan Dyer. 2022. Effect of scale on catastrophic
forgetting in neural networks. In International Con-
ference on Learning Representations.
Ruiyang Ren, Yuhao Wang, Yingqi Qu, Wayne Xin
Zhao, Jing Liu, Hao Tian, Hua Wu, Ji-Rong Wen,
and Haifeng Wang. 2023.
Investigating the fac-
tual knowledge boundary of large language mod-
els with retrieval augmentation.
arXiv preprint
arXiv:2307.11019.
Adam Roberts, Colin Raffel, and Noam Shazeer. 2020.
How much knowledge can you pack into the param-
eters of a language model? In Proceedings of the
2020 Conference on Empirical Methods in Natural
Language Processing (EMNLP), pages 5418–5426,
Online. Association for Computational Linguistics.
Chenmien Tan, Ge Zhang, and Jie Fu. 2024. Massive
editing for large language models via meta learning.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier
Martinet, Marie-Anne Lachaux, Timothée Lacroix,
Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal
Azhar, Aurelien Rodriguez, Armand Joulin, Edouard
Grave, and Guillaume Lample. 2023a. Llama: Open
and efficient foundation language models.
Hugo Touvron, Louis Martin, Kevin Stone, Peter Al-
bert, Amjad Almahairi, Yasmine Babaei, Nikolay
Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti
Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton
Ferrer, Moya Chen, Guillem Cucurull, David Esiobu,
Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller,
Cynthia Gao, Vedanuj Goswami, Naman Goyal, An-
thony Hartshorn, Saghar Hosseini, Rui Hou, Hakan
Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa,
Isabel Kloumann, Artem Korenev, Punit Singh Koura,
Marie-Anne Lachaux, Thibaut Lavril, Jenya Lee, Di-
ana Liskovich, Yinghai Lu, Yuning Mao, Xavier Mar-
tinet, Todor Mihaylov, Pushkar Mishra, Igor Moly-
bog, Yixin Nie, Andrew Poulton, Jeremy Reizen-
stein, Rashi Rungta, Kalyan Saladi, Alan Schelten,
Ruan Silva, Eric Michael Smith, Ranjan Subrama-
nian, Xiaoqing Ellen Tan, Binh Tang, Ross Tay-
lor, Adina Williams, Jian Xiang Kuan, Puxin Xu,
Zheng Yan, Iliyan Zarov, Yuchen Zhang, Angela Fan,
Melanie Kambadur, Sharan Narang, Aurelien Ro-
driguez, Robert Stojnic, Sergey Edunov, and Thomas
Scialom. 2023b. Llama 2: Open foundation and
fine-tuned chat models.
Ben Wang and Aran Komatsuzaki. 2021.
GPT-J-
6B: A 6 Billion Parameter Autoregressive Lan-
guage Model. https://github.com/kingoflolz/
mesh-transformer-jax.
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien
Chaumond, Clement Delangue, Anthony Moi, Pier-
ric Cistac, Tim Rault, Rémi Louf, Morgan Funtow-
icz, Joe Davison, Sam Shleifer, Patrick von Platen,
Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu,
Teven Le Scao, Sylvain Gugger, Mariama Drame,
Quentin Lhoest, and Alexander M. Rush. 2020. Hug-
gingface’s transformers: State-of-the-art natural lan-
guage processing.
Tianxing Wu, Xudong Cao, Yipeng Zhu, Feiyue Wu,
Tianling Gong, Yuxiang Wang, and Shenqi Jing.
2023. Asdkb: A chinese knowledge base for the
early screening and diagnosis of autism spectrum
disorder.
Tianxing Wu, Haofen Wang, Cheng Li, Guilin Qi, Xing
Niu, Meng Wang, Lin Li, and Chaomin Shi. 2020.
Knowledge graph construction from multiple online
encyclopedias. World Wide Web, 23:2671–2698.
Yunzhi Yao, Peng Wang, Bozhong Tian, Siyuan Cheng,
Zhoubo Li, Shumin Deng, Huajun Chen, and Ningyu
Zhang. 2023. Editing large language models: Prob-
lems, methods, and opportunities.
Lang Yu, Qin Chen, Jie Zhou, and Liang He. 2023.
Melo: Enhancing model editing with neuron-indexed
dynamic lora.
Wenhao Yu, Chenguang Zhu, Zhihan Zhang, Shuohang
Wang, Zhuosheng Zhang, Yuwei Fang, and Meng
Jiang. 2022. Retrieval augmentation for common-
sense reasoning: A unified approach. In Proceedings
of the 2022 Conference on Empirical Methods in Nat-
ural Language Processing, EMNLP 2022, Abu Dhabi,
United Arab Emirates, December 7-11, 2022, pages
4364–4377. Association for Computational Linguis-
tics.
Ningyu Zhang, Xin Xie, Xiang Chen, Shumin Deng,
Hongbin Ye, and Huajun Chen. 2022. Knowledge
collaborative fine-tuning for low-resource knowledge
graph completion. Journal of Software, 33(10):3531–
3545.
Yizhe Zhang, Michel Galley, Jianfeng Gao, Zhe Gan,
Xiujun Li, Chris Brockett, and Bill Dolan. 2018.
Generating informative and diverse conversational
responses via adversarial information maximization.
Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang,
Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen
Zhang, Junjie Zhang, Zican Dong, Yifan Du, Chen
Yang, Yushuo Chen, Zhipeng Chen, Jinhao Jiang,
Ruiyang Ren, Yifan Li, Xinyu Tang, Zikang Liu,
Peiyu Liu, Jian-Yun Nie, and Ji-Rong Wen. 2023. A
survey of large language models.
Zihao Zhao, Eric Wallace, Shi Feng, Dan Klein, and
Sameer Singh. 2021. Calibrate before use: Improv-
ing few-shot performance of language models. In
Proceedings of the 38th International Conference
on Machine Learning, volume 139 of Proceedings
of Machine Learning Research, pages 12697–12706.
PMLR.
Ce Zheng, Lei Li, Qingxiu Dong, Yuxuan Fan, Zhiyong
Wu, Jingjing Xu, and Baobao Chang. 2023a. Can we
edit factual knowledge by in-context learning?
Rui Zheng, Shihan Dou, Songyang Gao, Yuan Hua, Wei
Shen, Binghai Wang, Yan Liu, Senjie Jin, Qin Liu,
Yuhao Zhou, Limao Xiong, Lu Chen, Zhiheng Xi,
Nuo Xu, Wenbin Lai, Minghao Zhu, Cheng Chang,
Zhangyue Yin, Rongxiang Weng, Wensen Cheng,
Haoran Huang, Tianxiang Sun, Hang Yan, Tao Gui,
Qi Zhang, Xipeng Qiu, and Xuanjing Huang. 2023b.
Secrets of RLHF in large language models part I:
PPO. CoRR, abs/2307.04964.
A
Preliminaries of Model Editing
The task of knowledge editing is to effectively mod-
ify the initial base model fθ to the edited model fθ′,
with corresponding parameter adjustments for a
specific input-output pair (xe, ye), where xe ∈Xe
and fθ(xe) ̸= ye. Here, Xe represents the entire
set to be edited. Therefore, the current problem
formulation for knowledge editing can be broadly
categorized into three types:
1. Single Instance Editing: Evaluating the per-
formance of the model after a single edit. The
model reloads the original weights after a single
edit:
θ′ ←arg
θ
min(∥fθ(xe) −ye∥)
(1)
2. Batch Instance Editing: Simultaneously
modifying N knowledge instances (where N ≪
|Xe|) and evaluating the performance of the edited
model after processing a batch. The model reloads
the original weights after processing a batch of
edits:
θ′ ←arg
θ
min
N
X
e=1
(∥fθ(xe) −ye∥)
(2)
3. Sequential Editing: This approach requires
sequentially editing each knowledge instance, and
evaluation must be performed after all knowledge
updates have been applied:
θ′ ←arg
θ
min
|Xe|
X
e=1
(∥fθ(xe) −ye∥)
(3)
B
Default Hparams Settings
EASYEDIT provides optimal hyperparameters for
various editing methods. In addition to common
parameters such as learning rate, steps, and reg-
ularization coefficients, the location of layers for
editing can also be considered as hyperparame-
ters, significantly influencing the robustness of the
editing process. The following tables demonstrate
Layer with Value Loss
model.layers.31
Target Layer for Updating Weights
model.layers.5.mlp.down_proj
Table 3: Default Target Modules in ROME
Layer with Value Loss
model.layers.31
Target Layer for Updating Weights
model.layers.4.mlp.down_proj
model.layers.5.mlp.down_proj
model.layers.6.mlp.down_proj
model.layers.7.mlp.down_proj
model.layers.8.mlp.down_proj
Table 4: Default Target Modules in MEMIT and PMET
the default location settings in EASYEDIT (using
Llama-2-7B as an example).
ROME
We follow Meng et al. (2023) in utilizing
causal mediation analysis to identify an interme-
diate layer in the model responsible for recalling
facts. The causal traces reveal an early site (5th
layer) with causal states concentrated at the last
token of the subject, indicating a significant role
for MLP states at that specific layer (Table 3).
MEMIT
Following Meng et al. (2022), we quan-
tify the average indirect causal effect of MLP mod-
ules. The results demonstrate a concentration of
intermediate states in LLaMA. The disparity in the
effects between MLP severed and hidden states
severed becomes significantly reduced after the 8th
layer. We choose the entire critical range of MLP
layers, denoted as R = {4, 5, 6, 7, 8} (Table 4).
PMET
PMET (Li et al., 2024) adopts the local-
ization strategy from MEMIT, designating the cor-
responding layer as the modification target. Build-
ing upon the update of MLP weights, PMET fo-
cuses on multi-head self-attention (MHSA), further
substantiating the discovery that MHSA encodes
specific patterns for general knowledge extraction.
(Table 4).
MEND
In the context of meta-learning for edit-
ing, it is commonly observed that editing MLP lay-
ers yields better performance than editing attention
CodeBook Target Modules
model.layers[27].mlp.down_proj.weight
Table 5: Default Target Modules in GRACE
Target Layer for Updating Weights
model.layers.29.mlp.gate_proj.weight
model.layers.29.mlp.up_proj.weight
model.layers.29.mlp.down_proj.weight
model.layers.30.mlp.gate_proj.weight
model.layers.30.mlp.up_proj.weight
model.layers.30.mlp.down_proj.weight
model.layers.31.mlp.gate_proj.weight
model.layers.31.mlp.up_proj.weight
model.layers.31.mlp.down_proj.weight
Table 6: Default Target Modules in MEND
layers. Typically, MLP weights of the last 3 trans-
former blocks (totaling 6 weight matrices) are cho-
sen for editing (Mitchell et al., 2022a). EASYEDIT
adheres to this default configuration (Table 6).
GRACE
Recent studies have revealed the impact
of selecting the right layers for fine-tuning (Lee
et al., 2023). Similarly, in GRACE (Hartvigsen
et al., 2023), we conduct pilot experiments, retain-
ing layers with consistently high edit success rates
(Table 5).