Hey? Hi! My name's Talison Smith, and I'm with the that interactive simulations project at the University of Colorado, and several members of my team are here today.
Brett Fitler, and Emily Moore, Matt Moore, Jesse Greenberg, Ashton, are you here?
I'm here.
Okay, guys, tomorrow, so I don't know if I missed anyone.
We all work together to make flexible, interactive, multimodal interacts.
Yeah, multimodal interactive simulation at the that project at the University of Colorado.
And today we're not. We're not working on a simulation.
But we're working on a tool that we and this is mostly been done. This work is mostly been done. This work is mostly done by the rest of my team, but we're working on a tool that we hope to be able to use in co design a more conducive environment and
so it combines the we're calling it paper program.
I know paper, playground at the moment, and it's a tool that combines a database of programs that can be right presented on papers and also digitally as programs.
And and a camera view that detects the programs.
A projector that computes the programs and a database of program, and then an interactive board.
Where you can kinda see or see the brand, and maybe Brett wants to explain a little bit more about.
Yeah, I'd be happy to take over some of some of the any bits here. So Hi, everybody, Brett, also, as tell, yeah.
So we were. We've done a lot of work in the past when creating all the simulations that that we have and trying to build the sort of breadth of what we call inclusive feature into those simulations, and one thing has always, we you know, kind of continue to struggle
with is sort of the process of co-design with the variety of learners that we want to create, create with both the simulations and the features for the simulations and the timelines and the levels of kind of technical knowledge that are in the room
at the time right? Because I mean code design can look like any number of things.
It can be in the moments you have synchronous groups of people's an hour. Maybe it happens over and over again for us weeks or months.
Maybe it doesn't, and you know, as things are being designed right, the you know, there's different decisions being made at different times.
And a lot of the time, and when you're working, especially with some of the younger audiences that we alright, they they have lots of great, and they come up with these, and and we write them down, and we don't necessarily have a right. So we're.
Creating web effectively, web, development, right? Yeah, creating these things using HTML, csis Javascript as the main interactive component.
And not all of us are well versed know some more HTML and the like, but none of us are Javascript development, and and somebody has a really great idea. It'd be really great to in some cases at least, instantiate some part of
it or to be able to explain a little bit more, you know, not just talking back and forth about why, how this idea gets implemented, or how these these kind of work together, talking about a sound.
They really want to play. And we're also considering how that interacts with some description that's playing or some of the visuals that are happening at the same time.
And how some of these is your creating an interactive thing, how this can kind of cascade and that's hard to do with. Again.
Just language, or it's hard to describe an old thing with only one mode.
So we were, yeah, trying to think of different ways to sort of tackle this problem and kind of bridge.
The co designer, we'll bridge them together in sort of this design process.
So that LED us to this existing tool here. So I don't.
If with with dynamic land. Which was a was, is a kind of an experience happening over in Oakland, California.
But I was all around this idea of creating kind of like a collaborative space, and specifically there to do do programming.
But the idea that I was taken up by it's an open source project created by Developer Yan Paul Pasma.
And this paper programs.org and so you put this out here and we we're pretty smitten by this idea of not only kind of creating well, being able to take you, said a Javascript program.
So that's what Yann had created.
Here was a way to, you know. Write Javascripts and associate it with effectively a set of markets and to be able to put those out into the real world, and get them detected and have them run so basically running Javascripts
Javascript on paper. That was the selling point.
But what this meant to us was, well, one we're writing in Javascript.
So that's good, but it also rods brought the experience of making these changes and manipulating these kind of more abstract ideas that are, you know, our simulations represent whether those are kind of like physical equations you know.
gravity, or whatever we happen to be working on that day, and taking the control out of the screen, creating alright if it's like the programs themselves.
These scripts are a real piece of paper that you get. This scenario now tells, and hasn't in front of her, where there's a tabletop and we've and we've always been interested and strong believers that the physical space
is still very important, and in times where you can again have groups of alright designers and co designers in the same space by being able to take advantage of the physical space would would be very helpful.
So this tool as it was paper programs already captured. A lot of this.
And so what we wanted to you've been doing here is adapting that framework, that software within.
Yeah, kind of the co- design needs that we are seeing.
And so, yeah, so that kind of leads us to this.
And I think that after at this point we'll continue to talk more.
It's probably best to show talk extensively.
Right. So I just want to point out we're gonna I'll show a camera and then we're gonna load programs.
We'll do that with paper, because these programs are printing.
And and then we can like see the different effects that can happen, and Brett can edit.
Some programs live, and those will also make changes. So that's sort of kind of what we're gonna try and do here.
So just as a very simple, okay, so this is the program.
I'm sorry down here. That was the model for that.
Thing. So the when you come, when you load this thing up, you just get the supporting.
It. We have 3 lengths of camera, projector and design platform or interactive board.
Oh!
So it's in the camera view. You can load the papers sort of virtually. We're just gonna do a simple cuts down.
Hey? Hey? Telson? Yeah. Sorry. Can get back up just once more.
No!
Yeah, so, yeah, what tells us is gonna start to put out here is some examples that we already have some an important aspect about.
This is, you know, again, we're trying to have a kind of a thing to ease, to make code design sort of more productive for for all parties involved.
Part of that is, you know, kind of supporting us.
And so, yeah, we have again, not everybody's an expert in Javascript.
And we have developers with us. Right now, we're helping us work on the tool.
But one of the ideas here is that you know, we are creating these programs.
I do do different things. And we'll share some of the kind of simple cases of what you can do.
And the answer is pretty much anything you can write in Javascripts and push to the push through a web page is what you can accomplish with these, but yeah, retaining a lot of the kind of legacy functionality of paper programs.
And it again. I recommend post. But the programs that she's putting out here are all live in just a database, and this is one that we have hosted up so that we can, you know tell us in in Toronto now and me, I'm over here in
Philadelphia, and Emily Moore over in California, can all kind of work to and kind of create this.
These sets of programs that we can pull in and build upon.
And so, you know, creating more and more examples that are more useful to folks who are not necessarily developers.
So yeah, we'll just very quickly share a few of the kind of simple cases here and how it works.
And so every program is associated with a an actual physical paper, and it is associated with this kind of a series of colored dots.
There are 7 per corner, and they're colored black, blue, red, and green.
Each sequence is associated with the program.
So this this program here just sets the background color so you can see that it changed to blue.
Oh!
So this I can go in, live, and in in the code there and change it to.
If you wanted, and then this one should play a test sound, although I might sound off. Is anyone hearing anything?
No, I don't.
Did you share with sound?
I can't share with sound. Oh, let me see!
Stop share. That's fair. I didn't hear anything so share down, Eric.
Can you click into the board?
With this one I could.
So I don't know. I'm gonna get paid back.
No!
Not supposed to be.
Just be using.
Worked before. I'm not sure we did something. Well, during our last meeting that changed program.
Go ahead. I'm not sure I have sound, so. I wanted to play this down, but I'm not hearing it awesome.
I think so. You're muted.
Your computer's muted sorry. Alright! See in the top there's a.
Yeah, but I was old on mute. Try try on muting.
Okay.
Oops!
Yeah, that's muted and unmute.
I just had to mute the Collab room microphone because we were just gonna get some feedback there for me.
Telson, try unmuting on your zoom on your laptop right now try unmuting, and see if the audio will play.
It should be a zoom. It's hard, because I know Zoom moves its control panel off when you're screen sharing, you have to find that.
And unmute. Oh, there we go! Try that!
I heard it that time. Bye!
Hmm! I don't hear it.
Okay. Here. There's some feedback. Here's greeting with now.
Yeah.
Oh, I heard it!
I know. Yeah, just a simple.
Okay. Okay.
There it was!
Simple example. Taking advantage of web audio.
Okay.
Hmm! If only my business card did that when I handed it to someone.
So we're this guy fy over my AD bird is the working for feedback.
Cause. He's muted the volume and the room and then I guess I'm just talking with my computer.
Okay, so we have sound working. That's great. And then we have a simple text note that we can put on screen, which will come in handy for one of our examples later.
So these are just 3 super simple programs. Okay? Brad changed the color to purple.
The last kind of the blue, and how we can't hear that very much anymore.
Turtle, and we're looking.
So those are a simple example. The system is working, the dots are being recognized.
Sometimes lighting is problem. Then we can use a more interesting example.
Yeah, here I'll pause here real quick.
Yeah, and so the goal. Obviously, if we were working side by side with someone or I was in the room with Tilson, and we would turn out to troubleshoot something here is that we could either you know, on subver computers be working through the editor and Tab over to the editor there. Real quick that's the window that I have open on my
screen.
Oh, yeah.
Yeah, I mean, even see that I'm currently in it, and she can't access it.
But that this is the kind of thing that we'd be working with.
And so I just wanted to dwell on this for just a quick moment, because an important part here is the idea of, you know, again getting non technical on Javascript of helpers involved in the process here, one of which I said, of course was, you know, the more examples there are the more things that are already. Here.
The more people can just drag and drop, and sort of make small changes to, and part of that process at the moment in, you know, right now, sort of the Javascript structure is, you know, this that kind of idea of templating and adding extensive documentation in
code, so it's just really clear what are the what the things?
That you might want to change if you were, if for whatever context it's like you're trying to make.
So here, if we're trying to just play around with some sounds, you know, we would know that.
Telson. Okay, sorry. Can you scroll back up to the top?
That it would be fairly easy to change just you know the file that's being played.
Sorry. Yeah. Your editor is kind of small at the moment, but hmm!
Yeah, no. Problem.
Yeah, and you could kinda iterate on this process.
And assuming that these things are sort of well documented, and it's clear what you can change that there's gonna be can be quickly, quickly modified right in the moment for the person you're working with.
Yeah, I'm not sure where the file is, but there is a name for okay.
Oh, it's at the top. It's just scrolled over. But that's okay.
Okay.
We can. We can go back to the.
Okay.
Now we have the whole field here. So those are my 3 simple programs.
And then we have other examples that could be with altitude.
Yeah, and and I'll start. And just the city a little bit.
So just sharing the simple stuff you know, if folks are looking at this and starting to get a sense for what's doing and what you know, seeing some of the kind of more generic capabilities and thinking about, you know what possible use cases you could have
we're gonna show a couple of examples here that start to do the kind of thing that we were.
Like, I kind of mentioned before. You know again, we're in these cod design moments, and you kind of want to see the interrelatedness of the things that you're designing, that you're across across different modes. So you know here.
We'll highlight visual sonification, and how those play, depending on the kind of you know, underlying relationships that we have.
So again, the kinds of relationships that we're usually dealing with are of the physical equation.
Nature, often some sort of STEM concept. That's a physical equation.
Chemical equation, mathematical relationships which many things boil down to.
Even if you were to use a you know, financial models, some sort.
But yeah, so you know, kind of highlighting here.
And this first one is fairly simple and fairly linear.
But.
A more specific use case that we consider, and what we're aiming for and using it.
But but it could be generalized or useful, I would say, of the STEM contacts as well.
Hmm, and then this one, this paper here is just adding that image.
You start to see it at the bottom. Now, this next program is called Change Altitude, with paper.
So I'm gonna add this program, and I'll be able to use the paper as a as well.
At low altitude. You are a treetop flyer.
And it's voicing which didn't expect.
But that's nice.
It's I think it's recognizing the program in the left edge of the screen.
Oh, okay, perfect. I'll just add that in, because that's the voiced content.
At sea level. You are in the splash zone.
So you could work with people to come up with different descriptions here, and you could just listen and group paper.
At low altitude. You are a tree top flyer at high altitude.
You are a stratospheric sore at very high altitude.
Telstra, and can you pull the image out?
You are at the edge of space.
Sorry.
Can you pull the image out?
Hold image! Out! Yup!
Hmm! You might need to move a little bit further over.
Oh, not on the port! Oh, sorry!
Yeah, so.
At high altitude, you are a stratospheric, sore at low altitude.
You are a tree top flyer.
Yeah, so in a lot of the work. Uhhuh, a lot of the work that we do, you know, we that's some of the most challenging parts of the code design are outside of the visual it's the relationships of the sounds.
And and and and how does play together? That is kind of the hardest for us to prototype a lot of the time.
And so just to emphasize here that, like the kinds of examples that the prototypes that we come up with, don't require a visual.
Hmm!
There's still a number there, but we don't have to be there and tell us a good play with this, and if you could throw in the sound program.
And people hear that.
Yup!
Yeah, we can hear.
At high altitude at very high altitude. You are at the edge of.
That'll change a volume, I think, as a gets higher, lower.
Pitch.
High altitude. You are a stratosphere at medium altitude.
You are a cloud crew at sea level. You are in the splash zone.
Yeah, you can play with it. And knocks like the sound with the description and the to see if the like.
The model that you're it's been kind of described or split accurately through the description.
I who sounded at the same time and not me. This takes several iterations.
If I'm going okay, I would try this, and I give it to Jesse.
Can implement. And then I have to go. Listen to it again.
After I thought about it, and then you know, and where is this? I can kind of just do it live, or I can do it.
Live with another, and do it virtually too.
Sorry. Just wanted to put that in there, Brett, you were saying.
Yeah, yeah, so there's pretty much maximum flexibility here.
And there's some some more nuance that we can talk about, in terms of how we, how we structure sort of what parts were building onto this gang, because a lot of this tool existed before.
What we're building. That supports the kind of design and the kinds of things that we're trying to create.
And a lot of that has to do with making things fairly robust with.
You know the introduction and like when things disappear and when they don't sort of what happens.
So when the thing, the program here, which is the which is just the thing, that sort of defining what?
What is altitude, and how does that change that?
It's yeah what happens when that it disappears and nothing else can see it anymore.
But things don't just break, because that would be frustrating of.
At low altitude. You are a tree top flyer.
Yeah, I. Have. I have a question about this this collaboration is really fascinating to to see live, because, having gone through that process as well, I'm trying to create text descriptions or audio descriptions for things that are happening live and just sort
of that development bag behind like design and development and testing, and like all that is really fast.
That all shortened because of through this through this kind of interaction.
I'm I'm just kind of curious.
If there has been any, if a teacher would be able to access the same tools and be able to customize the descriptions, and very much the same way, cause I'm thinking, because fed simulations are often appealing at different grade levels, your description, say you might want to provide might be
different, depending on the on the student as well, so will the teacher be able to access the similar tool, be able to customize their audio description, so that they have something more customized personalized for their vert.
Yeah, so that's actually, probably a very good question.
That's related to a lot of the other work that's ongoing with server fols in production.
At the moment we are kind of focusing on this as one on sort of simpler interactions.
The goal at the moment is not to support the ability to create like a full simulation, if only because they are very at least the things that we're creating are very complicated in terms of all the moving pieces, to get that.
But when you know, when we're actually designing things, the like, the types of interactions that we're kind of like fussing over and trying to even just grab our brains around can be made in sort of like a smaller subset of the of the
people, so like, you know, rather than having entire thing about Ohm's law, right?
Voltage Equals Current Times. Resistance for those who may may remember or have to a circuitry, and the like.
Electronics is sorry. Is that? Yeah? Not having to create the entire experience. There.
But that okay, well, we know the relationship there. And right now tell us, just really trying to figure out how, with a learner at the time, what's a better way to have that mapping between what's being said and like maybe the the scale of it's the
range across the values, whether or not actually. And this is a good point here to show off another thing is right, not everything is always just linearly related, right?
They're not one to one mappings, but things can be inversely related.
Things can be exponentially related, and those things get even harder for us to think about.
But yeah, so that we said, You know, for publication, ready simulations.
That question has a very different answer. I'll say, from this point, that's yeah.
We'd love for eventually. Right? If this is a deployable tool and we're able to share the database of examples because it is important here that the database itself is at the moment a remotely hosted thing for us.
But you know we have a copy of it copied in a github, so anybody could could have those, but they have to either choose to put it up remotely somewhere themselves, unless we're hosting it.
That kind of server, or they're able to do it locally.
It's just a little extra setup at the moment.
But yeah, absolutely that they'd be able to create their own setup.
So they wanted to, you know, have these sort of simpler, simpler experiences.
Some kind of interactive experience with what kind of interaction.
So I kind of see it more like designing like an interaction.
And then some of those interactions, I mean being able to design an interaction in this kind of space would then help me sort of like, refine the design which would be part of the simulation.
But certainly with the school I would be able to refine the language much more quickly on a more iterative process.
Or then my current process, which is like billing tables, and then translating that the developer, and then having an implement and then having to test it alright.
So so this I can listen to in advance, like, for example, like sometimes I need a really simple scale, like a few morning, get charges and positive charges, charges several morning.
The charges and positive charges like, I think I have 5 on scale, for the number of charges and loses that electricity.
But when describing distance and around the course of basics, I think I used to 7 point scale, and then to come up with the words in terms of you know, very close to, you know, extremely close to you know, extremely close to, like you know, and and those are related to other things about
life before, so making sure that those description then mock to what's happening visually, and the other tech the other.
Description, that you hear while you're making that refraction.
So sometimes you just want it. Test out the scale of which is kind of hard to do with description, easier with sound.
Yeah, yeah, since we think we'll want time for more extensions.
But doing the same, and then you can do them together like sound, and the description together generally don't do unless the sound, unless the sound gets implemented first.
But let's maybe move through a couple more examples.
Okay.
Quickly, here, so let's go back to the altitude really quickly.
Okay.
You already have that open and just cause? Yeah, because John brought up Ro the question and then the native mappings came up.
That could be kind of important. So I'm gonna switch I'm gonna it's actually we'll have you do this.
So we're trying to hide anything about the editor here.
There's no magic happening. Go ahead and remove the control.
Oh!
So, yeah, an important distinction here is that the actual for the for us, like the altitude model the thing that's defining the relationships between things right?
If it's saying this, this is altitude.
And that's the value of altitude. And you guys can see at the moment there's a 0 on the straight the board, and it increments with the value of of the altitude.
Hmm!
But that's being controlled by entirely different program and not by the model itself.
And so there's a, you know, again, ability for the programs, the papers to talk and send values out and share them across and make use of them.
So you saw some of them change sort of like the display.
So they're listening for the altitude value to either voice it out, or play the sound or update the sound.
And but that can be flexible.
So. Oh, sorry! Let me not be honest, so you can edit it.
Okay, well, I always just let you edit it. I was just gonna show it.
But you need to edit it for.
Okay, you you can open it backup.
Okay.
Yeah, so altitude, model is so just to kind of show.
You know how things can change. If the that the mapping is different, and the kind of thing you might want to explore.
And so this is a simple change. But oh!
Alright. Make this a bit quieter. Lots of user interface improvements to make alright.
Okay, so yeah.
Yeah, so, and you can see some where we're at right now with the ability to edit document up some of the papers.
So this is again a the code associated with that paper that Tellyson has out, and kind of fixing most of the things that people might want to change near the top.
And one of the things that we've noticed for designers such as ourselves is developing the Javascripts is pretty hard, but finding the plate things that we can change is not quite as hard, so as sort of an intermediate step yeah this is kind
of where we're reaching. So yeah, okay, sure, we can try that.
So switching from a you know one to one mapping to more of a more of a inverse relationship and just being able to quickly change that kind of thing. Alright cause.
Those are some of the most. Yeah.
Okay.
Sort of challenging ideas to wrap, wrap your head around things that are outside of the have linear space.
Okay. So oh, we're gonna show the board. Maybe.
Sorry. Awesome this over.
Okay, so and I worked instead of having the paper changed, it.
I changed it so the model paper and changed the position.
And I'm playing the paper down.
At high altitude. You are a stratospheric, sore.
And it's going off. So I just.
At low altitude at sea level. You are in the splash zone.
So you can. So I've inverted the model here.
Change it to excellent.
Yeah, that's fine for a time. Let's jump over to denzens.
Okay.
So, yeah, just a 1. One more quick demo here to show again.
Like a step, a step more towards more more sophisticated.
And some of these programs the designers themselves were able to create, based off the existing templates and little snippets of code that are kind of well documented.
Voice mapped, value.
New mass, value 5.0 kg.
Here, let's actually just put. Can you remove the density model?
New mass, value 5.0 kg.
Yup. You want to pull it, pull it out real quick.
Okay, so density models in there.
Hey? What? You wanna pull out the density model real quick.
Oh, okay.
Yeah, just to highlight one other possibility here, that, yeah.
Again. What does the thing we're gonna show it here is that like this idea of being able spatially map things to the thing you know, the mappings that you're trying to control feels feels pretty good in general, it's pretty novel and fun.
But also, yeah, again, we alright. Really hopeful and convinced, is this helpful collaborate environment, but not everything has to be tied to those.
Nor you know sometimes your physical spaces and strange. So there's also kind of like mock paper virtual representations that can be placed in here.
Hmm!
Alright!
So if you wanna just put the density model in there since it doesn't depend on anything that you might want to change, you know, spatially.
That's you. Don't wanna take up your own physical space.
So this is this is a paper as far as the program's concerned, and does everything that it would do if it was being detected like the other papers.
But you can sort of you know. Move it off the side, and not take.
Take up your space. So our density models now in there, and that defines our relationship, that we've added is the text over there.
So density is mass over volume, so volume is inversely proportionate.
So when volume goes up, density goes down, mass goes up, density.
Okay, so density, volume, value.
New volume value, 1.0 leaders.
And I can put in the image.
And change massive paper density, voice, density values.
Sure!
Density, 5.0 kg per liter density, 5.0 kg per liter.
I would probably remove voice volume at the same time.
Voice. Volume. Oh, okay.
Yeah, your top?
Density, 5 kg per liter density, 5 kg per liter.
Okay, so this is gonna change. Bye?
I have density model in there, and I have density volume.
And I have image. Matt.
Yeah, we can change the masks. And so one of the other points here is that there's more that you can take advantage.
So any of those marker positions you can be used to kind of make this visual relationship so that can be a lot of things that we've shown so far.
Just sort of like the center position of the paper. So maybe you're vertically changing it or horizontally changing it.
But here we have a maybe a slightly obscure mapping of a paper area to volume, which might be a strange concept.
If you if you take that, but yeah, so I'll larger area means just a larger volume.
So the bigger. That's that paper gets the higher the volume, the lower the density, and a simple mapping of transparency on the image to density.
So we'll lower the density more transparent.
The image, and so you start to see how all these things can kind of fit together.
Yeah, some sounds in there. At the same time, there's different options for her voicing density value as well as the volume or the mass.
What changes of transparency!
It's mapped to that city. So if you could take out the mass so the kind of thing that people are gonna do if they're playing around with density is, you know, raise and lower the mass phrase lower the volume and see what effect it has on density, so you could
click the control mass with paper and move that up and down.
That's a paper.
Control mouse with paper.
The one thing that we are finding at the moment is that in addition to finding all of your your files and scripts, you're now also need a fairly decent organization system for your physical programs.
Okay.
Maybe folders, perhaps.
Little actual folders.
Okay, so this is changing my ads of that barrel. I wasn't quite sure what the volume did. Oh!
Yeah. So yeah, making a larger volume will makes the density go down if it's large enough.
Oh, okay.
But the the kinds of things that you run into are.
Yeah, it does the system. So a lot of kudos to to yawn.
Who created paper programs because he figured out all this code, sorry that allowing things to overlap and still be recognized.
New volume value 1.6 liters.
New volume value, 1.0 leaders.
Alright!
And so you could also imagine tying things to your rotation and anything else that kind of happens with a TV plane.
As it is, so that brings a yeah, I'll probably we've a lot of this here for now and go to questions in a moment.
But we, and it's again. There's some nuance that's not really.
I'm not hoping not nuanced in the long run, but sort of what we're trying to support.
Here is the idea of, you know, particular programs that represent the relationships between the things that you care about.
So that's that was models. You have what we call, you know, the view. So the different displays, whether that's the voicing sound. And then things that control stuff, right?
And typically, the power of this is to have them map spatially and not necessarily just virtual components, though Tellson could toss in the slider if you wanted to and share that if when you're thinking about ui components.
New volume value 1.2 litres.
You. You can do that as well and interact with the Slider on the board over there.
Yeah, you can even do it.
New volume, value 2.6 litres.
And so, you know, part of the power is that we're trying to adapt this to fit with our own.
That library fed Api, which supports a lot of our inclusive design features.
So you know interacting with the board automatically gets you things like the highlights.
Yeah.
And some of you know our voicing can be stopped with the control button, and a lot of the ultimate support.
So, yeah, our specific use case there.
But yeah, making sure that what is what we're interacting with is faithful to what we're trying to create.
And I lost, lost my brain there for a moment.
Oh, yeah, sorry. So, yeah, sort of splitting between those different types of programs.
And what goes into the actual kind of support that?
But what structures go into the actual software to make that kind of easier to create and understand.
For those who are working with it. And so I just want to, very quickly, you know, indicate what we definitely know.
We're going towards. So you know, there's always the possibility of kind of making the code sites simpler in terms of who's interact with.
So the way I've sort of mentioned is that we're really going for like more examples, more templates, things very well documented down the line.
If whether or not it's us or this, this is all open source, and I'll drop the the the repo in in the chat here in a second.
That's you know, other through collaborators, or that additional features are added in terms of yeah, what the code looks like.
Does it get abstracted? Kind of like dynamic land which is kind of like a Lewis scripting.
So making just like the types of things that you are typing a little higher level.
So they're easier to understand. Do you? Going towards things like block coding, which is a lot more approachable to folks?
Not, you know, inined in the syntax of Javascript, to the different features for the detection.
So you know, things are currently pieces of paper in 2 dimensions are clever and Eisenberg here and with us we're working to kind of understand where we can push that and what what supports we can have in the tangible world here in terms
of craft components and things to touch and move and recognize.
It's a boost. Both the collaborative and just sort of tangible experience of prototyping.
Oh, sorry! Those are no little prototype. You have questions there and then, of course, you know, for us to start really working with the populations that we do.
We have worked and improve both our process co design process as well as the tool itself.
Yeah, and so, yeah, there's a few questions here.
Alright! I think I've kind of asked them throughout this, but you know ours is, there's very some specific.
So if anybody sees value outside of that, things that you see about the tool that are exciting or very confusing, and the like.
Lisa has a question.
Hi lisa!
My name's Tson. Hi, I have a couple of questions.
This is incredibly cool, I think that the potential for this for co-design, especially with kids, is really exciting for me to see.
And I can definitely see up, even in some of our projects where we do co-design with kids.
I wish separately was here. She did a lot of codesign for a weavely project and definitely voice the concern around technology in some cases being a barrier for co-design for kids and wishing there was something more tangible that we could bring into the
activities. That's where I see one area of application.
For sure I have 2 questions. One is really simple, and I couldn't quite see the cards themselves, so it's hard to know what information is on to what extent those paper artifacts can be changed so like, can you draw pictures inside?
Or like. Are there any implementations in terms of what might interfere with the limitations in terms of what might interfere with the ability of the program to scan the actual code so that you could also include things like icons and pictures illustrations and words and then my.
Well, let's find out.
Awesome. Great question right now. I have in bigger text the names and numbers and programs.
Yeah.
So so off, I think in the original design of this paper programs, the code is printed there. But it's not. That's not how we we've got that.
If you can't call it.
And yeah, sorry you had a so I'll I can demo something if you don't ask that. Your other question.
Sure!
Tales and type still screen, share real, quick.
Yeah.
As so mayor, other question is around, kind of two-way collaboration of this could be a part of so how can.
So I guess to smaller questions, as part of that one is, would it be possible to have also a digital interface that is similar to the paper interface?
So that you could have a kind of like collaborative touch board.
And then I'm wondering if you've given any thought to what it might look like to have two-way collaboration.
And I'm thinking specifically, if you have, let's say one code is participant who's interacting with the physical.
And then one co-design participant, that interacting with the digital version, how it would be possible for them to talk to each other, whether through like projection or something like that.
Yeah, so the way the tool setup now is, you know, fine, large, the kind of collaborative aspects through the database.
And so you know, the tool is constantly talking to it.
If a program gets updated, then it changes it for everybody and the tool itself, right?
So I'm running the same thing as Telson is right now, and on my side we're connected to the same database.
But you know I have my own individual thing running here, and it is doing its own thing, its own thing right.
So it is separate the moment, as far as putting thought into that.
Yeah, it's it. It has come up because, Eric, the more that we can start to think about.
Yeah, sort of this version, remote, remote co- design is its own.
It's own challenges with its own, and she's better.
We're a remote team. So tell us and I are trying to use this to do something.
Then, you know, being able to do some of the things that you're answering here.
Hmm!
So having basically this shared shared view that you see here, it's a shared kind of play space by area that's running the same programs would be great.
It might be. It's something we'd love to work on.
I think.
Well, I mean one thing we did notice is like I can play with the like.
Say if one of us has a physical setup and can play with the physical setup for lack of a better word, the virtual paper, they will obviously moving the virtual paper will change the values of the model, but it won't necessarily change.
It won't change the physical with with the tool.
So if someone wanted to make it make a physical change to the model, not the program, but what the information that's being representative, they could do it with paper, or they can do it with the virtual paper.
That that square that we just saw that was dragged out.
That was like a virtual representation of a card. I see.
Yeah.
Yup!
Okay, very cool. Thank you.
Yeah. So at the moment, that's something that we wanna do actually, in the near term is, have the kind of mock or virtual papers aligned more closely with the physical experience.
And that includes some of the things that you can do, that you can't do with the current program.
Sorry. The current recognition of of the virtual right.
They're rectangles, but the things you can do with real pieces of are.
Oh, these are on the wrong order. Bottom, right top rate.
Sorry. Is extend just a single alright. I can just pull it.
One create, you know your arbitrary quadrilaterals, I suppose?
As well as you know. Rotate with that which of that is not currently capable of?
So having that more faithfully captured that, and you know possibilities for gesture sort of a computer vision control of the virtual bringing, the embodied.
Manipulation to the virtual side to in cases where you know, physical setups aren't possible aren't easy, Jonathan.
Yeah, I have a question more, I guess maybe it's a two-part question.
The. So I'm curious about the hardware requirements.
The technology requirements for actually running this, the reason why, I asked this is we've also had projects where we've worked with with groups in either remote or has low access to technology.
So even having cards printed on color might be an issue, so black and white might be like a laser printer, might be the only option, or even just access to the hardware is another.
Is it might be a barrier, so I'm just curious like, how can this scale?
How does this scale into a different context, where they access that technology would be different.
Yeah. So they'll part of that is similar to the to the previous previous comments.
So a lot of the resource. Intensive parts of this are using Opencv, the open source computer vision software to do all the recognition.
So if you were to not use the physical papers.
But we're doing the virtual that could theoretically cut down a lot on the resource intensiveness.
So we are, you know, attempting to try different levels of webcam quality and computer quality or performance to see how it works across those.
But right so one is, you know, if you can't print papers, that there's still a pretty good experience.
Still a collaborative, helpful experience, virtually is part of our AIM, but also again, kudos, again, to pay for programs.
And it's developer that's the recognition is pretty good. If you use markers.
If you have a piece of paper, and you just color those dots that's actually how I started, because I didn't wanna use my incup in our very early iteration.
But yeah, it actually works surprisingly well, just to have.
For you know these at the moment. These colors, black, green, red, and blue.
So the program will framework like a show. You a representation.
Great thoughts. Yeah.
Hey? Yeah, I don't know how long these are to go for, but we're happy to stick around and answer questions.
Sure. Well, I guess I I could help wrap it up.
Typically these meetings run for an hour. But thank you. Thank you all for coming.
This is really fascinating. Just sort of like bridging that gap between.
You know, digital prototyping and physical prototyping and being able to iterate.
So thank you so much for coming and sharing this, and I hope you all come back again.
And just you know, demonstrate again that sort of like the next advancement.
If people wanna hang around they can hang around. But if you have to run off, you know, feel free to go.
Do that as well. So thank you. Everyone for coming today.
Thank you, Tyson and Brett. It's great. Thank you.