Welcome!
Understanding Loss Functions, Backpropagation, and the PyTorch Transformer Understanding Loss Functions, Backpropagation, and the PyTorch Transformer
In the field of deep learning, loss functions and the backpropagation algorithm play a crucial role in training neural n
2023-06-22 data science
deep learning loss function backpropagation
How to choose the right batch size in deep learning such as Transformer How to choose the right batch size in deep learning such as Transformer
Choosing an appropriate batch size in deep learning, including models like Transformer, requires careful consideration a
2023-06-20 data engineering
transformer deep learning batch size
Fast way to map an array X and array Y using nonparametric regression, a decision tree based regression Fast way to map an array X and array Y using nonparametric regression, a decision tree based regression
If want to find a a non linear relationship between two arrays, and it’s hard to model it using simple functions such as
2023-06-19 data science
regression descion tree
What does start and join mean in multiprocessing in Python What does start and join mean in multiprocessing in Python
In Python’s multiprocessing module, the start() and join() methods are used to control the execution of a Process object
2023-06-19 data engineering
multiprocessing
Understanding Parallelism in Python, Threads vs. Processes and concurrent.futures Understanding Parallelism in Python, Threads vs. Processes and concurrent.futures
In Python, parallelism is a technique that allows programs to execute multiple tasks concurrently, thereby improving the
2023-06-19 data engineering
multiprocessing
Utilizing Human Feedback for Enhanced Fine-Tuning of Large Language Models, RLHV vs finetuning difference Utilizing Human Feedback for Enhanced Fine-Tuning of Large Language Models, RLHV vs finetuning difference
Wonder what’s the difference between RLHF method with finetuning method? Fine-Tuning is actually the first step of a RL
2023-06-17 data science
finetuning RLHF
Constraining Prediction ranges in Deep Learning Regression Models Constraining Prediction ranges in Deep Learning Regression Models
Deep learning is widely used for various regression analysis tasks. However, sometimes we need to apply certain constrai
2023-06-16 data science
deep learning regression
New Function role added to GPT3.5 and GPT4 to give model to use API or functions as tools New Function role added to GPT3.5 and GPT4 to give model to use API or functions as tools
The latest update (June 2023) from OpenAI added a new role called function into their tool box.By describing functions i
2023-06-15 data science
chatgpt openai gpt4
The short history of open source large language model from chatGPT so far The short history of open source large language model from chatGPT so far
Large language models (LLMs) have revolutionized the field of artificial intelligence, and their long-lasting impact is
2023-06-13 data science
large language model
Handling Data Type Mismatch between NumPy Arrays and PyTorch Tensors Handling Data Type Mismatch between NumPy Arrays and PyTorch Tensors
In NumPy, the default float data type is float64, also known as double precision. In PyTorch, the default float data typ
2023-06-12 data engineering
pytorch
Input dimension for pytorch transformer when the sequence length is one Input dimension for pytorch transformer when the sequence length is one
In pytorch, when we want to provide a 2-dimensional input with shape (batch_size, embedding_size), we should be careful
2023-06-11 data science
pytorch transformer
How critical is to normalize the input data, since there is layer normlization in the transformer encoder already How critical is to normalize the input data, since there is layer normlization in the transformer encoder already
Normalizing input data is generally a good practice, as it can improve the convergence rate and the overall performance
2023-06-11 data science
pytorch transformer
13 / 28