Skip to content

Commit

Permalink
Edits to concepts, use cases and ethics pages
Browse files Browse the repository at this point in the history
  • Loading branch information
aishaeldeeb committed Aug 20, 2024
1 parent 9a863d2 commit 461e83c
Show file tree
Hide file tree
Showing 3 changed files with 13 additions and 16 deletions.
12 changes: 4 additions & 8 deletions content/concepts.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,12 +45,10 @@ This process is common in fine-tuning tasks because it ensures that the model's
**Embeddings** are numerical representations of words or tokens in a continuous vector space. They are crucial in enabling models to understand and process language. Unlike one-hot encoding, which represents words as discrete, high-dimensional vectors, embeddings capture semantic relationships between words in a dense, lower-dimensional space.

### How Embeddings Work
- **Dense Vectors**: Each word is represented by a dense vector of fixed size, where each dimension captures a different aspect of the word's meaning.
- **Semantic Similarity**: Words with similar meanings are represented by vectors that are close to each other in the embedding space.

1. **Word Embedding Basics**:
- **Dense Vectors**: Each word is represented by a dense vector of fixed size, where each dimension captures a different aspect of the word's meaning.
- **Semantic Similarity**: Words with similar meanings are represented by vectors that are close to each other in the embedding space.

2. **Example of Word Embeddings**:
- **Example**:
Let's consider the sentence "The cat sat on the mat."

- **Original Words**: ["The", "cat", "sat", "on", "the", "mat"]
Expand All @@ -63,13 +61,11 @@ This process is common in fine-tuning tasks because it ensures that the model's

In this example, each word is mapped to a vector in a high-dimensional space (e.g., 100-dimensional). The vectors capture semantic relationships: words like "cat" and "mat" might be closer to each other in the vector space compared to "cat" and "sat," reflecting their related meanings in context.

3. **Learning Embeddings**:
- **Learning Embeddings**:
- **Training Process**: Embeddings are learned during the training of models such as Word2Vec, GloVe, or BERT. The model adjusts the vectors based on context and co-occurrence patterns in the training data.
- **Contextual Information**: Modern models like BERT produce contextual embeddings, where the representation of a word changes based on its surrounding words. For example, the word "bank" will have different embeddings in "river bank" versus "financial bank."

### Positional Encoding


**Positional Encoding** is a technique used in transformer models to provide information about the position of tokens in a sequence. Unlike recurrent neural networks (RNNs) or convolutional neural networks (CNNs), transformers do not have inherent mechanisms to understand token order. Positional encoding helps address this by encoding the position of each token in a way that the model can incorporate into its processing. Positional encoding injects order information into the input embeddings to ensure that the model understands the sequence of tokens. This is crucial because the transformer architecture processes all tokens simultaneously and lacks the sequential processing of RNNs.

**Example**:
Expand Down
5 changes: 5 additions & 0 deletions content/ethics.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,10 +33,15 @@ The rapid rise in AI raises profound ethical concerns:

3. **Lack of Explainability and Transparency:** LLMs are built on complex neural networks that perform extensive processing. Neural networks are often considered "black boxes," making it difficult to understand how deep neural networks make their decisions.


## Final thought:
When using LLMs as writing or research assistants, it is very important to consider how they can affect one's skills and ability to produce original work. It is essential to understand that the more we rely on LLMs to do complete or assist with our writing or researching tasks, it can diminish our skills and increase our reliance on such tools. Therefore, it is best to use LLMs only when necessary or when working with deadlines and tight schedules. To maintain and improve on existing skills such as writing, coding, or researching, one must deliberately practice constantly and attempt to produce original work.

## Read More At

1. [UNESCO Ethics of AI](https://www.unesco.org/en/artificial-intelligence/recommendation-ethics)
2. [Conceptualizing AI Literacy](https://go.exlibris.link/fFY0XLFX)
3. [https://medium.com/@luiz_araujo/chatgpt-will-kill-your-writing-135576ae9655](ChatGPT Will Kill Your Writing)

---

Expand Down
12 changes: 4 additions & 8 deletions content/use_cases.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,7 @@ nav_order: 6

# Use Cases for Large Language Models (LLMs)

Large Language Models (LLMs) offer a broad range of applications that can significantly impact various research fields. These models can assist researchers in multiple ways, from automating repetitive tasks to generating insights from large volumes of data. Here’s an overview of how LLMs can support research across different domains, including both engineering and humanities.

## Applications in Humanities Research
Large Language Models (LLMs) offer a broad range of applications that can significantly impact various research fields. These models can assist researchers in multiple ways, from automating repetitive tasks to generating insights from large volumes of data.

1. **Text Analysis and Interpretation**
- **Historical Document Analysis**: LLMs can analyze historical texts to identify themes, sentiments, and linguistic patterns, providing insights into historical contexts and literary trends.
Expand All @@ -22,17 +20,15 @@ Large Language Models (LLMs) offer a broad range of applications that can signif
- **Creative Writing Assistance**: LLMs can help writers brainstorm ideas, generate content, and refine drafts, supporting creative projects in fields like fiction writing and screenwriting.
- **Cultural Analysis**: They can analyze cultural texts and media to uncover trends and societal changes, contributing to research in cultural studies and anthropology.

## Applications in Engineering Research

1. **Automated Literature Review**
4. **Automated Literature Review**
- **Summarization**: LLMs can process and summarize extensive scientific literature, helping researchers quickly grasp the essential findings of numerous papers. This can streamline the literature review process, allowing researchers to focus on key insights.
- **Semantic Search**: By understanding the context and meaning of research papers, LLMs can enhance search engines to find relevant articles based on nuanced queries, improving the efficiency of literature searches.

2. **Data Analysis and Insights**
5. **Data Analysis and Insights**
- **Pattern Recognition**: LLMs can analyze large datasets to identify patterns or anomalies that may not be immediately apparent. This can be particularly useful in fields like materials science or bioinformatics.
- **Predictive Modeling**: They can assist in developing predictive models by analyzing historical data and making forecasts, which is valuable in engineering disciplines such as systems modeling and reliability engineering.

3. **Natural Language Processing (NLP) Tasks**
6. **Natural Language Processing (NLP) Tasks**
- **Code Generation and Review**: In software engineering, LLMs can help generate code snippets, suggest improvements, and even perform code reviews, which can accelerate development processes and enhance code quality.
- **Technical Documentation**: LLMs can automatically generate or refine technical documentation, making it easier for engineers to maintain up-to-date project records.

Expand Down

0 comments on commit 461e83c

Please sign in to comment.