Skip to main content

Conference: Multiplier Event 4: Open Education

At Open Education Hub, we have been working on creating a methodology, developing the support infrastructure, and populating digital content repositories, using the open source collaborative model. We want to capture feedback on these items, to raise awareness, and to encourage teachers and trainers to use them.

So, on Wednesday, September 13, 2023, 09:00-16:00 EEST, we are hosting a multiplier event / conference to present our approach to developing and deploying digital educational materials as Open Education Resources (OER). As part of the event, we are organizing practical workshops to get a hands-on experience of our work, and brainstorming meetings to get your input on important questions regarding the future of education in the digital age. Our goal is to extract best practices, guides and lessons learned, in order to build up the methods, approaches and infrastructure for the future of education, a future that will be tied to digitalization and technology.

The event takes place in the Central Library of University POLITEHNICA of Bucharest.

We will start with a conference style set of presentations, continue with practical workshops, have lunch and then do brainstorming sessions. We will have coffee & drinks freely available.

If you are an educator with interest in improving the quality of educational materials and the process of using them to teach, evaluate and support learning, this is an event you want to be part of.

If you plan to attend, please fill this form.

Schedule​

Agenda​

  • 09:00-09:15: Welcome
  • 09:15-10:45: Talks / Conference, room 2.1, 2nd floor
    1. Open Education Hub: overview, goals, vision, values, results so far
    2. Methodology for Open Education
    3. Infrastructure for Open Education
    4. Repository walkthrough
  • 10:45-11:15: Break: 2nd floor foyer
  • 11:15-13:00: Workshops (bring a laptop): rooms 2.1, 2.2, 2.3, 3.2
    1. Creating and deploying an Open Education Hub repository (room 2.2)
    2. Automating assignment evaluation with vmchecker (room 2.3)
    3. Adding linters and GitHub actions in your educational repository (room 2.1)
    4. Developing evaluation content with Open Education Hub (quizzes) (room 3.2)
  • 13:00-14:00: Lunch: 2nd floor foyer
  • 14:00-15:30: Brainstormings: rooms 2.1, 2.2, 2.3, 3.2
    1. Attracting and engaging students during live activities (lecture, seminars) (room 2.1)
    2. Statistics and metrics for the teaching and learning processes (room 2.2)
    3. Using Open Education Hub methodology and infrastructure beyond Computer Science topics (room 2.3)
    4. Developing, maintaining and assessing assignments and projects as OERs (room 3.2)
  • 15:30-16:00: Brainstorming conclusions and parting thoughts

Conclusion​

Open Education Hub: Overview, Goals, Vision, Values, Results so Far - RăzvanD​

RD: Today we will show what we've done, and we'll collect feedback from you. How will technology affect education? We want to make teaching great again, it should be more about mentoring, and less about repetitive tasks.

RD: There's a lot of bureaucracy when teaching that can be automated easily. Content development shouldn't waste time each year, we should only improve what we have. We want to have everything already there.

RD: We use the OER guides in this project. We started with a few people, we gathered some more along the way and now we want to gather even more. Iteration makes perfect.

Methodology for Open Education - RăzvanN​

RN: Educational content should be free and available to everyone. There are creators and consumers. The methodology treats these aspects.

RN: I'll talk about the content creation, and Sergiu will talk about the infrastructure. One will go to the Open Education Hub GitHub and will find lectures/content about biology, history, etc. We present guidelines on how to create content, how to update it, and how to use it.

RN: We structure content into the following: reading, media, guides, drills, slides, and projects. Drills are simple exercises, while projects are broader. You need to use more topics and information to solve them.

RN: We then state what content types we need to use for a specific activity. The activities are lectures, practical sessions, assignments, self-study, tests. The mapping can be one-to-one or one-to-many. For example, for lectures, multiple types of content can be included (e.g. slides, media, drills, guides).

RN: Find the repository structure in the template. We chose this structure so one can choose what chapters one would teach. We keep the entire collection of chapters and you can choose the topics. Ideally, a script will extract whatever you want and will publish it to a website.

RN: As for our next steps, we create these tools to do all these stuff automatically, and extend the methodology for other topics.

Infrastructure for Open Education - Sergiu​

SW: Servers go brrrr. Setting up classes takes a lot of teachers' time. Another issue that we have is integrating feedback.

SW: Our goal is to make the teacher's job easier. We want to focus more on the interaction with the students, and less on setting up classes. We want to automate everything.

SW: We want to build content that is OER. To do this we have to have the content editable by regular users, mostly text format files: slides, tasks, assignments, quizzes. The slide you are looking at is actually a Markdown file.

SW: The OER builder receives a configuration recipe, and the content, and it servers an HTML content. We use HTML because it's text and easy to host.

SW: We use Docusaurus, which converts Markdown into HTML easily. Reveal-md generates slides from Markdown files.

SW: After building the content, we want to distribute it. By using HTML and git, we have the advantage that this content can be viewed in the browser. Automatic publishing is also part of this flow. We can publish content on both GitHub and GitLab.

SW: Others will find the content and they can suggest improvements. The suggestions come as pull requests, the owner of the repository is responsible for "accepting" the suggestion, but apart from that, everything is published automatically. SW: After building and publishing the content, we need to check the students' assignments. They need instant feedback. So we built VMChecker. It uses containers for easy checking.

SW: We integrated VMChecker with a familiar interface - Moodle. Students can upload assignments via GitLab or Moodle, both of them allowing them to receive instant feedback from VMChecker on Moodle. A teacher creates an assignment in Moodle and that's it.

Repository Walkthrough​

Operating Systems - Teo​

TD: We have all the content in the Operating Systems repository. It's rendered using Docusaurus as Sergiu has just explained.

TD: Some of the things to note about the content are first that slides are embedded into the website like so. These slides are stored in Markdown format in the repository on GitHub and then "compiled" and rendered on this website so anyone can browse them.

TD: Another thing to note is related to our assignments. We have 5 up to now and we're planning to keep adding assignments so that at some point we may choose a given number of them at the start of the semester and use those for evaluation. All assignments are graded via the new VMChecker that Sergiu has also touched on.

TD: Now moving back to the repository, RăzvanN told you that students are encouraged to contribute. We award cryptocurrency rewards for this, namely SmileyCoins. Initially, we're doing this manually, but for tracking, we're also assigning labels to these PRs. Being just a token coin, the amounts we're awarding are symbolic: 1 and 5, depending on the PR's complexity.

RD: Keep in mind that the rendered website is produced automatically from the GitHub repository.

TD: Yes, when a contribution is made to the GitHub repository, there is an automated system in place that publishes the updated version of the website. This way, students always have access to the most up-to-date materials.

Security Summer School - Gabi​

GM: We migrated the SSS to the OpenEduHub format. Now we'll see a "before and after" comparison. We started with the wiki page (2014-2020) where we had to manage access to people that could contribute and there was no way of reviewing changes.

GM: Afterwards we switched to an intermediate format, not so different from the current Open Education format. It used Hugo, a theme for Docsy. We had some problems with rendering different types of content.

GM: Right now we have SSS on Open Education Hub. We follow the methodology that RăzvanN talked about, we have multiple chapters with different types of content.

Computing and Calculus for Advanced Statistics - Eggert​

EKH: I teach a mostly self study course at University of Iceland. We had this material, it was tough to maintain, the person responsible for the content is about to retire.

EKH: Our task was to take this content in LaTeX form and create Markdown from it. We thought about using automatic conversion tools from LaTeX to Markdown, but each tool we tried required a lot of configuration.

EKH: Originally, students created parts of this content, each had a different style, different structure, some had images. EKH: After lots of trials, we got it to work.

Questions​

Marius: Is there a nicer looking content already available?

GM: We have something nicer in the Operating Systems course.

Marius: Can you for example modify the background?

RD: Yes. We will integrate all sorts of renderers that allow this. We didn't have the time yet for this.

Cătălin: Do you have the option to change the theme?

RD: Yes, reveal-md allows this. We propose here more of a methodology, the outcome itself is now not the best it can be, but it works. We can make it look prettier.

Cătălin: Will the system support raw HTML?

RD: Yes.

Dragoș: Let's say I'm a student, the teacher created a content, can I download the content?

RD: Yes, everything is in a repository.

Bogdan: What you presented are entire courses. Could it work for some educational content (not an entire course).

RD: There is going to be a central repository that contains all of them. Afterwards you make a copy/fork and take the course. Any sort of teaching content should be here. The process should apply anywhere. Children, highschool teachers, trainers, anybody. Based on the config, you choose what to extract from an entire topic.

Costin Carabaș: Do you have support for different languages?

RD: Not currently, we want the content to be internationalizable. Now it's in English because it was the most appropriate one to start with. Some might complain about this, but in the future, we will make all items internationalizable.

Adi Stanciu: How do you cover overlapping content? RD: It's not obvious. Someone should not create a new class simply because you want things differently there. On the other hand, if you need content from multiple classes, you specify this in the config.

Adi Stanciu: How do you help teachers select the proper content for the class? Let's say you have content, different levels of detail, and conflicting ideas.

RN: The idea here is that you have the chapters organized on topics. You go on the topics, you have more reading files, drills for them and select the ones that are fit for you.

Lucian: How much time did it take to migrate the OS content to this format? Maybe this could benefit from automation.

RN: It's hard to tell because we didn't have all the content and then we deployed. We built the content from zero, so it's different. It's not publishing the content that is complicated. The most time-consuming part is to think through new ideas and how to structure things, what to add, what to remove, what to avoid, and how to make it better.

Radu: How does this format handle team assignments?

RN: Only one can submit.

Workshops​

Creating and Deploying an Open Education Hub Repository - Vlad​

VN: Issues arose for the participants using Windows. The suggestion was to add a section on the oer-template repository for instructions on how to use Docker Desktop with WSL on Windows in order to develop content.

VN: There was also a question about LaTeX support for math classes, but that is already covered in the openedu-builder.

VN: There were some questions about self-hosting the generated content behind a reverse proxy (Nginx / Apache). Since the output is a static website, it can be pulled from the GitHub repository and put behind a reverse proxy (or the content can be built locally and published).

RD: Infrastructure is useful to be used outside the educational environments. There were people who were looking to easily update their website. It’s a non-trivial problem to do website updates. You could go for wikis (but they are rather content centric). Static site generators (SSGs) such as Docusaurus (see list here: https://jamstack.org/generators/) are very useful for this. And they can be easily backed by GitHub, i.e. using the collaborative model.

RD: People will not work with Git in the command-line. But the Git web interface is excellent for editing. A flow that isn’t concerned with Git, but a web interface or overlay on top of Git would be good.

RD: Based on the point above, we need “tooling” - interfaces that ease up the creation, management and update of content sites (not necessarily educational). Quite a lot of people would benefit from

  1. being able to do it (it’s not that easy),

  2. having the benefits of Git / GitHub, collaboration, version control, multiple items, reviews.

Automating Assignment Evaluation with VMChecker - Mihai​

MB: Before the presentation, I was asked if there are any actual examples of assignments that were used in production. I pointed them to the SO repository.

MB: Presentation about vmchecker

MB: Demo for setting up testing infrastructure

AM: Most institutions in Romania use Moodle, so it was a good choice for support with the plugin

AM: Do you integrate with the Moodle repository?

AM: When Moss will be integrated do you plan on comparing against solutions from the previous year?

Adding Linters and GitHub Actions in Your Educational Repository - Gabi + Alex​

GM: Using an already configured GitHub actions pipeline makes it easier / faster to change your content repository. Using linters to always keep the same structure for your content.

GM: I did a demo part presenting how GitHub Actions works, and how you can configure other GitHub actions. The same was for linters, with some examples on how to solve linters errors.

GM: We ended the presentation by doing a full PoC demo: creating a new organization, for the template OER repository, enabling GitHub Pages and deploying a new website using GitHub Actions.

Developing Evaluation Content with Open Education Hub (quizzes) - Ștefan + Teo​

SJ: We use Markdown to store the questions for quizzes, since it’s easier to write and render (both on GitHub and other frontends). Other platforms might require a different format (e.g Moodle requires Moodle XML) for the questions, so we want to provide a simple script that can convert between some known and used formats.

Vlad Traistă-Popescu: Does this script support images? SJ: No, not yet.

Vlad Traistă-Popescu: We did something similar, it uploads the image as `base64`` and Moodle knows how to handle those.

Multiple types of questions (not just multichoice) should be supported.

The workflow is still quite technical (cloning a repository, running a python script, importing stuff into Moodle). The user should not even have to deal with the idea of "question format", it should be a drag-and-drop.

Attracting and Engaging Students During Live Activities (Lectures, Seminars) - Liza​

Cosima Rughiniș: name tags for students and random questions directed at them during the lecture.

Cecilia: working in groups, but there's always a group where only one guy is working.

LB: This is good for seminars, but not so much for lectures.

CR: Analyzing a dataset based on the answers of the audience.

LB: Analyzing quiz answers and explaining/asking why/what they got wrong

LB I find it difficult to pay attention of a larger discussion.

DB: Multitask doesn't work.

RR: There is a sweet spot between number of items one is working on: too much multitasks vs too much boredom (due to not so many items to work on).

TB: There are two parts: passion / genuine interest, and the other one is fear.

DB: The use of Kahoot. It as a competitive metric.

JK: My university tried the flip classroom.

CR: University encourage people to appreciate their differences, particularly in social sciences.

OM: Stop patching the system instead of coming up with a new ideology.

CR: Students learning to coordinate. Of course, the students may forget a lot of what they learned. But coordination is important. I would say the most gain of a university is this interaction. You're not ready for the job market.

LB: Can we have a common way to get more people engaged during classes?

DB: I was more attracted to a topic taught by a teacher I like.

MA: The idea would be to personalized learning.

CC: My goal when teaching is for them to understand what I'm saying. And how to make them think correctly about economics. Trade off between

CR: Maybe rather than speak about needs, make students learn about items is meaningful. It has to be have meaning.

LB: People matter. Content matters. The relationship between students and professor

ES: In academia, we tend to use metrics. How do we incentivize students to think about the "why" - the reason why something it's relevant?

Quizzes and asking questions keep students engaged.

TB: Not focus so much on getting their attention, but rather on not losing their attention.

TB: Feedback is important and knowing when to do it.

CR: I think repetition is really valuable. Because some people may be lost after 5 minutes.

ES: For the first year classes, any teacher could implement it.

RD: Consider what you can do or what you cannot do. There are system issues: number of students, jobs, load.

LB: How do you measure engagement?

CC: If I get a good, then they're engaged.

ES: If the process is there, the answer doesn't matter.

DB: You could use something like that TikTok to hook students in.

There is a division in themes.

RD: Engagement is relevant, but patience and perseverance are important.

DN: What we can do is aim to have a friendly environment.

RD: Yes, aim to not disengage students.

RaduD: The most loved students are those with more open attitude. Such teachers seem more human.

ES: The sentiment of being in the boat together.

ES: You want to emphasize internal metrics.

TB: The real educators make sacrifices of their R&D time.

Statistics and Metrics for the Teaching and Learning Processes - Alex​

After setting up the OpenEducationHub repository, it's essential to consider how to assess its effectiveness. The main topics we will discuss will be about:

Collecting Metrics and Feedback​

At the most fundamental level, the gradebook serves as the primary form of feedback, indicating how well students are performing academically.

In addition to grades, one approach to get insight on how well the subject is going is to implement an end-of-year feedback form that allows students to express their overall satisfaction or agreement with the subject or course. This feedback form can be a valuable tool for gathering qualitative insights into students' experiences, opinions, and perceptions of the subject matter and the educational approach.

In addition to the annual feedback forms, setting up weekly feedback forms is essential to promptly capture immediate or "real-time" feedback. These forms serve as an effective means to detect moments when students may lose interest or encounter difficulties in comprehending the course materials:

  • Brief and Focused: limit the questions to 3-4 to encourage completion.
  • Direct Questions: use closed-ended questions to save time.
  • Optional Expression: forms are a way for students to share thoughts and are not mandatory metrics.
Processing Feedback​

Raw feedback isn't always ideal because it can be emotionally charged, potentially affecting you personally. One approach to address this is to have a third party review and filter out aggressive or non-constructive feedback, leaving behind only valuable suggestions for improvement.

However, this filtering process can result in a loss of information, making it harder for you to fully comprehend certain opinions. To address this, consider providing students with the option to include their email addresses in the feedback form. This enables you to reach out to them later for clarification and a deeper understanding of their perspective.

In essence, this approach closes the feedback loop and encourages engaged students to contribute more effectively by facilitating a two-way conversation for better communication and comprehension of their feedback.

Using Metrics​

Analyzing correlations between feedback and grades can provide valuable insights into necessary changes: When students who have received lower grades in specific areas consistently provide similar feedback, it signals potential deficiencies in the educational materials for those areas. When students with high grades also provide positive feedback, it suggests that increasing engagement for other students may be beneficial. Conversely, if most students express positive assessments of the subject but exhibit divergent grades, it raises the possibility that the provided materials may not be sufficient for a complete understanding of the concepts.

Feedback itself can be used as a transparency tool. This involves sharing with students the ideas and suggestions provided by previous student cohorts and engaging in discussions about the improvements that have been made in response to this feedback.

Using the Open Education Hub Methodology and Infrastructure Beyond Computer Science Topics - RăzvanN​

One person attended, a professor from a Faculty of Geography. He is teaching a course on creating and using maps (physical and digital) to students that don’t have a background in computer science or using programming tools such as git. He was very interested in how the methodology can be applied in his case, given that there are multiple universities that teach such a course, each with its own version of teaching materials. Using the our methodology to develop education materials for such a course would bring great benefits to both the teachers of the course (they can share maps which are not easily found on the internet and are hard to update) and the students (that can access older versions of the maps that are being presented).

Takeaways​

Open Education Hub Repository - Vlad​

  • Issues with Windows users highlight the need for clear instructions, such as using Docker Desktop with WSL for content development.
  • Acknowledgment that infrastructures like static site generators (SSGs) and GitHub-backed systems are valuable beyond educational settings.
  • Emphasizing the need for user-friendly interfaces to facilitate content creation and management, reducing dependency on Git's command-line.

Automating Assignment Evaluation - Mihai​

  • Demonstrating real assignment examples in production, like the SO repository, can illustrate the tool's effectiveness.
  • Acknowledging the importance of integration with commonly used platforms like Moodle for wider acceptance.

Adding Linters and GitHub Actions - Gabi + Alex​

  • Employing GitHub actions pipelines streamlines repository changes and maintains consistent content structure using linters.
  • Practical demonstrations, like setting up a new organization and deploying a website using GitHub Actions, aid understanding.

Developing Evaluation Content with Open Education Hub - Ștefan + Teo​

  • Addressing the need for a simpler process, like drag-and-drop functionality, to ease technical workflows for creating quizzes.
  • Simplifying the script for converting between different question formats and incorporating image support for enhanced functionality.

Engaging Students During Live Activities - Various Presenters​

  • Diverse strategies discussed, including student engagement via name tags, group work dynamics, quizzes, and analyzing audience responses.
  • Highlighting the importance of personalized learning, meaningful content, and fostering student-professor relationships for better engagement.

Metrics for Teaching and Learning Processes - Alex​

  • Stressing the value of timely feedback, including both end-of-year and weekly feedback forms, to evaluate the effectiveness of educational methods.
  • Addressing the need for a refined feedback process to filter and understand suggestions better, while encouraging two-way communication.
  • Using student feedback and grades correlation to identify areas for improvement and sharing past feedback as a transparency tool.

Using Open Education Hub Beyond Computer Science - RăzvanN​

  • Exploring the potential application of the methodology beyond computer science, such as in geography courses, emphasizing the benefits of sharing educational materials among different universities.