update: look at todo
parent
960219f412
commit
032de4e7f4
|
@ -33,6 +33,6 @@ def answer(query: str) -> tuple[Any, list[str]]:
|
||||||
return answer
|
return answer
|
||||||
|
|
||||||
|
|
||||||
print(answer("Who is Rishi Sunack"))
|
# print(answer("Who is the author of TinTin?"))
|
||||||
|
|
||||||
# def custom_answer
|
# def custom_answer
|
||||||
|
|
|
@ -32,6 +32,16 @@ def GET_GOOGLE_API_CONFIG() -> tuple[str, str]:
|
||||||
return (GOOGLE_API_KEY, GOOGLE_SEARCH_ENGINE_ID)
|
return (GOOGLE_API_KEY, GOOGLE_SEARCH_ENGINE_ID)
|
||||||
|
|
||||||
|
|
||||||
|
# TODO: work in progress
|
||||||
|
# class GoogleAPI:
|
||||||
|
# def __init__(self) -> None:
|
||||||
|
# self.GOOGLE_SEARCH_API_KEY: str = ""
|
||||||
|
# self.GOOGLE_SEARCH_ENGINE_ID: str = ""
|
||||||
|
|
||||||
|
# @property
|
||||||
|
# def google_search_api_key
|
||||||
|
|
||||||
|
|
||||||
def NLP_config(mode: str = "default", debug: bool = True) -> None:
|
def NLP_config(mode: str = "default", debug: bool = True) -> None:
|
||||||
global NLP_CONF_MODE, CONF_DEBUG
|
global NLP_CONF_MODE, CONF_DEBUG
|
||||||
CONF_DEBUG = debug
|
CONF_DEBUG = debug
|
||||||
|
|
|
@ -1,6 +1,4 @@
|
||||||
|
In this paper, I present {\bf \inlp} a new control-flow wrapper abstraction to enable the utilization of data from the internet (or a knowledge-database when offline) for existing context-needing Natural Language Processing (NLP) models to function without any given context. Internet-NLP can be used, finetuned alongside existing NLP models via its config settings and additionally its Long Short Term Memory neural network (LSTM neural network) can also be trained. Additionally incorporations of Masked Language Models (MLM) such as BERT, or LinkBERT \cite{devlin-etal-2019-bert,yasunaga-etal-2022-linkbert} can be utilized to improve search queries, and therfore retrieve more accurate and reliable data. Futhermore, {\bf \inlp} utilizes a LSTM, Reinforcement Learning and caches to allow for multi-turn NLP tasks, and improvement via Reinforcement Learning from user.
|
||||||
|
|
||||||
In this paper, I present {\bf \inlp} a new control-flow wrapper abstraction to enable the utilization of data from the internet (or a knowledge-database when offline) for existing context-needing Natural Lnaguage Processing (NLP) models to function without any given context. Internet-NLP can be used, finetuned alongside existing NLP models via its config settings and additionally its Long Short Term Memory neural network (LSTM neural network) can also be trained. Additionally incorporations of Masked Language Models (MLM) such as BERT, or LinkBERT \cite{devlin-etal-2019-bert,yasunaga-etal-2022-linkbert} can be utilized to improve search queries, and therfore retrieve more accurate and reliable data. Futhermore, {\bf \inlp} utilizes a LSTM, Reinforcement Learning and caches to allow for multi-turn NLP tasks, and improvement via Reinforcement Learning from user.
|
|
||||||
|
|
||||||
|
|
||||||
Additionally in this paper, I also present new NLP and Natural Language Inference (NLI) models to assist {\bf \inlp}:
|
Additionally in this paper, I also present new NLP and Natural Language Inference (NLI) models to assist {\bf \inlp}:
|
||||||
|
@ -22,7 +20,4 @@ As a result of these models, datasets, and Internet-NLP, the accuracy and reliab
|
||||||
|
|
||||||
Internet-NLP and the new NLP and NLI models, which were trained on the general-purpose datasets (ALotNLI, and ALotOpenBookQA). Internet-NLP, by default utilizes an Text-Generative model GPT-NeoX \cite{gpt-neox-library, gpt-neox-20b} for long responses and LinkBERT \cite{yasunaga-etal-2022-linkbert} for short responses. For 2 choices (for ex: True and False) Bi-Encoder NLI has been used and for multiple choices CrossEncoder will be used \cite{thakur-2020-AugSBERT}.
|
Internet-NLP and the new NLP and NLI models, which were trained on the general-purpose datasets (ALotNLI, and ALotOpenBookQA). Internet-NLP, by default utilizes an Text-Generative model GPT-NeoX \cite{gpt-neox-library, gpt-neox-20b} for long responses and LinkBERT \cite{yasunaga-etal-2022-linkbert} for short responses. For 2 choices (for ex: True and False) Bi-Encoder NLI has been used and for multiple choices CrossEncoder will be used \cite{thakur-2020-AugSBERT}.
|
||||||
|
|
||||||
Internet-NLP, in layperson terms, provides the context for context-needing NLP models to let them function. Internet-NLP can be improved via finetuning, and training of LSTM and Reinforcement Learning model (which can be trained alongside the NLP model), which enables for better search queries, and subsequently results. It obtains state-of-the-art (SOTA) results in QA and NLI without context.
|
Internet-NLP, in layman terms, provides the context for context-needing NLP models to let them function. Internet-NLP can be improved via finetuning, and training of LSTM and Reinforcement Learning model (which can be trained alongside the NLP model), which enables for better search queries, and subsequently results. It obtains state-of-the-art (SOTA) results in QA and NLI without context.
|
||||||
|
|
||||||
Internet-NLP is a subset of a larger package, Internet-ML and is open-source. $\footnote{Internet-NLP, subset of Internet-ML is public, and open-source: \url{https://github.com/thamognya/internet_ml}}. \label{footnote:code}$
|
|
||||||
Old versions of Internet-NLP is also publicly available. $\footnote{Old Versions of Internet-NLP is public: \url{https://pypi.org/project/internet-nlp/}}. \label{footnote:code-old}$
|
|
||||||
|
|
Binary file not shown.
Loading…
Reference in New Issue