WARP – Week 6

Captain’s log, stardate: -303819.

As we have finished all of our main lab work, this week was just correcting plots, taking a closer look at some of our bodies and getting started on the group report.

Two of our previously shown figures have been updated, one of which is the heatmap. With this figure, the errors were fixed so that they are now the correct orientation as previously they were not.

The final heatmap used to find the metallicity of the bodies now with the correct errors.

The other updated figure is the number density plot, now with better-labelled axes and errors on each data point.

A plot of number density per spectral type now with errors and better-labelled axes.

In the previous blog post, it could be seen that we had some bodies that we thought could be metal-poor stars. We looked at these stars, as well as others that we were not sure were galaxies or stars, in a piece of imaging software called DS9 in which you can look at the image with a greater contrast to observe its shape more easily. It was concluded that all of the stars shown last week, shown again below, are stars apart from WARP-1 and WARP-9 which cannot be classified as stars or galaxies solely from their shape. This means they cannot be ruled out as they could still potentially be metal-poor stars. As well as these, there are also two more potentially metal-poor stars; WARP-40 and WARP-57.

Images of bodies which could potentially be metal-poor stars.

Other than this, we have just been making progress on the group report where sections have been designated to each member and we aim to have a first draft at the beginning of next week.

WARP – Week 5

Captain’s log, stardate: -303838.

The Theoreticians

Battered, beaten and severely malnourished by a ship-wide tea shortage, morale among the crew was low, but their noble mission was finally coming to an end.

We plotted a new graph of T against g-i now including O and B stars. We found that there are two lines of best fit on this graph, one for low g-i values and one for high g-i values. In TOPCAT we added a subset that grouped bodies with high g-i values (g-i>-0.4) and found that all but one of our stars fell into this subset so we did not think it was necessary to calculate the line of best fit for the one star in the lower g-i portion. We found that the fit that was best for the high g-i values was a fifth-order polynomial and so created a new column in TOPCAT which calculated the temperatures for our bodies using this equation. With this temperature we created subsets for each star type using the temperature ranges mentioned in last week’s blog.

The crew gazed upon these purest of Milky Way travellers, in awe of their magnitude. Nothing could have prepared them for their sheer elation at the accomplishment of their goal. Starfleet would remember this.

As the Coding Wizards had extended their heatmap, we were given updated metallicities for our bodies and matched these with the data we already had to create one table with all values in. Using this table, we sorted by metallicity and created a naming system of decreasing metallicity named “WARP-” where the most metal-poor body was WARP-1. Using COSMOS Cutouts, we got images of the first 65 bodies (metallicities of less than -3) by inputting their RA and Dec. We attempted to determine whether they were galaxies or stars and found some potentially metal-poor stars!

Here is WARP-1. We know it’s not a single star due to its shape but it could possibly be a binary system and as it was our first (star) baby, we love it the most (don’t tell the others).

An image of WARP-1, what we believe to be a binary star system.

Below are our four best candidates for extremely metal-poor stars. They are all more circular bodies compared to a lot in the dataset, meaning they are more likely to be stars than galaxies which are typically shaped more like an oval.

Images of the four best candidates for extremely metal-poor stars.

The Coding Wizards

We wished to include black bodies in our theoretical plot as this would allow us to further the range of metallicity values used to create our heat map, however reliable and consistent results in NB392-u-(g-i) were not able to be produced. The black body results in g-i however are consistent with what is expected and as such this leads us to the conclusion that an error must arise due to either/both of the NB392 and u filters. A leading theory is that the similarity of said two filters and the lack of any absorption in the model of black bodies is causing the discrepancy.

We completed a refined heat map using the np.polyfit Python function to more clearly define the boundaries between the different metallic items on the plot. The np.polyfit function allowed us to extrapolate our data points such that the layers of metallicities that was expected could be successfully programmed in our code.

By comparing our actual data to the heat map, as shown in below figure, we obtained new values of metallicity and the errors in them.

The final heat map, the method we used to determine the metallicities in our observed stars.

This plot, while slightly unclear, greatly illustrates how our code can determine the possible metallicity of an actual star from the colour change of its magnitudes, by analysing its position upon this heatmap.

We have also created a plot of the number density of stars with respect to their spectral type, showing how dense stars of different metallicities are. The plot is shown below, however work still needs to be done to incorporate errors.

A plot of number density per spectral type where the y-axis is logged for convenience.

We also aim to soon (hopefully) rectify the error in the modelling of black bodies, such as possibly plotting the heatmap with another colour axis, to hopefully remove the error if it is in fact caused by the similarity in u and NB392.

And with that, WARP’s mission was complete. The crew reflected on their journey, a journey fraught with chaos, misery and excitement intertwined, and could not help but be saddened by its end. Picard admired his crew in a way only a captain can: with happiness, pride and an overwhelming sense of faith. Whilst he could not be sure of the voyage upon which they would next embark, he could be safe in the knowledge that his loyal crew would be right there at his side.

Note from Star Trek guy: That’s all from me – thanks for reading! I hope you’ve had as much fun reading these useless little paragraphs as I’ve had writing them. – James (Cdr. William T. Riker)

Note from the communications lead: I wanted to leave this blog saying how nice it has been writing this each week to summarise what we have accomplished. Even though for the past five weeks we have been unsure about what to do on many occasions as well as coming across many errors in our work, it is very satisfying to conclude we have finally found some bodies which we believe to be metal-poor stars! We may be posting over the next few weeks about figures or report writing but here is the end to all of our lab work! – Claire (Cdr. Deanna Troi)

WARP – Week 4

Captain’s log, stardate: -303860.

The Theoreticians

After analysing the error horde’s level of infiltration into the data, a shocking revelation occurred among the crew. It wasn’t the errors the team could see that was the nightmarish thought but rather the ones they could not… La Forge saw fit to limit his ship’s exposure to error-infested space. 

By plotting a histogram of the original dataset’s magnitudes in the NB392 filter (shown below), we chose a limiting magnitude to the data of 24.71 which is the peak magnitude. This meant that the volumes of the sky we are observing for each spectral type had to be recalculated and that the data that has a larger magnitude than this can be excluded as this data is incomplete, containing stars too faint to observe. This then meant a new subset had to be created to further reduce the number of potential metal-poor stars by only including data points with a magnitude in NB392 lower than the limiting magnitude.

A histogram of the NB392 values from the original dataset used to find a limiting magnitude.

The entire crew gazed proudly upon the vast celestial abyss they had spent so long examining. They had all come so very far.

We also plotted all of the original data on a graph of RA and DEC overlaid with our reduced catalogue of potentially metal-poor stars. This showed how much data reduction we have done over the previous weeks to try and find potentially metal-poor stars in a section of the sky containing over 120,000 bodies.

A plot showing the position of all bodies from the original dataset in the sky overlaid with bodies that are potentially metal-poor stars.

Guinan strode confidently into the room. She seated herself at a terminal and got to work, eager to show the crew just how much this humble bartender was capable of.

The rest of the lab session was spent organising and obtaining data to create a plot of stellar number density against spectral type for a variety of metallicity brackets. To create this histogram, we have to classify all of the data points in the reduced catalogue as a star type. As we have been using g-i the past few weeks, we plotted a graph of data from the Coding Wizards of temperature against g-i. By plotting a line of best fit on this graph, we can find its equation which can be used to determine the temperature of all of the data points we have. We plotted the graph below, but this does not contain any B stars. We will plot these on the graph next week as this will increase reliability of the line of best fit for all stars.

A graph of T against g-i created using data provided by the coding team to generate a line of best fit which can then be used to find the temperatures of our potentially metal-poor stars.

When we have the temperature of all of the stars in our reduced catalogue, we can then classify all of them using the temperature ranges below.

The temperature ranges used to classify the stars. Taken from Dr. Sobral’s PHYS263 lecture notes.

The Coding Wizards

Almost as if some divine creator itself were tormenting them in some kind of sick irony, a new, very different kind of error reared its ugly head before Picard and Data’s eyes.

We began working on being able to calculate the metallicities for any stars placed onto the heat map. The first included some issues with using the data that had blank spaces. As TOPCAT and Windows read blank spaces differently, it caused the code to crash every time a blank appeared, as such we placed a mask on the data to ignore any point that included a blank. The next task was to ensure that all the data fitted within the scale of the heatmap. If the point did not lie inside the grid we created, it would result in an out of bounds error. Originally, if this was the case, the metallicity was set to -99 which would easily show that the point was not valid.

Using information obtained by Riker and the away team, Picard and Data once again harnessed the power of their enemy against it – the only way to defeat a foe as formidable as this.

Next the errors on the metallicity were calculated. By taking the error for the magnitude in each filter (which is specific to each data point) a random variation on that point was generated from a gaussian distribution. 1000 randomly gaussian distributed points were created and then the mean and standard deviation were calculated from the metallicities found at these points. This gives one metallicity and one respective error for each of the original data points.

However, this caused problems as the points could go out of bounds for some of the errors and not others. As such, the metallicities could end up around -50 with an error of +/-50. Realising that this was an error, the out of bounds metallicity was set to not calculate anything and ignore the point. As such, the stars that did not fit had no metallicity calculated and those whose errors fell outside the plot had more accurate metallicities calculated.

We are now able to calculate and output the metallicities for many different types of stars with correct errors.

Real sources with error bars placed on a metallicity heat map to visualise their metallicity.

Data tirelessly (tiring was merely a human flaw) worked his way through more data than any human could hope to handle in their lifetime. His objective was to do in a matter of hours what no human could ever hope to accomplish. A task so monumental in magnitude that its sheer size made even him unsure of its outcome…   

We also attempted to improve the accuracy and quantity of the theoretical points that allowed us to generate the heatmap in the first place. By doing so, we could hopefully have a larger scale and thus calculate the metallicities for more stars. This proved to be a much larger challenge than initially predicted.

WARP – Week 3

Captain’s log, stardate: -303877.

The Theoreticians

Riker’s hard work was finally paying off as Picard and Data now had the tools to combat their daunting task: overcoming the error hordes by fighting fire with fire. It had worked countless times over millennia of conflict – Picard could not help but reflect on the poetic futility of endless interstellar war.

We started off by finishing calculating errors that were started last week for the Coding Wizards to use to generate clouds of theoretical stars. This was done by creating a subset of the average errors in the x-axis and y-axis of their graph and finding their average value using a function within TOPCAT. Histograms of these errors were created to show where the peak errors were within the data, an example of which is shown below.

A histogram of the errors in the y-axis of the Coding Wizards’ graph for all of the data points given.

Riker had had enough. He hoped to immobilise the error threat by removing the most threatening within their ranks, in effect, cutting the head from this most dangerous of snakes.  

As we needed to narrow down our data, we had to filter out extreme values and pick a reasonable limit to the error in both axes using the histograms. We started with a maximum error of 0.5 in the y-axis and 0.15 on the x-axis but when we plotted this data on top of the coding team’s produced data, we had too many potential extremely metal-poor stars (stars beneath the coloured cloud).

An initial plot of bodies overlaying the theoretical clouds produced by the coding team.

Some errors had slipped past our brave away team. They were dealt with swiftly and efficiently, but with a sense of restraint and control only seen among the most composed Starfleet officers.

After placing error bars on this graph, we realised we had used the unfiltered data which still contained errors outside of our range, as well as points with no errors, so we plotted the correct data of stars with errors within our error range. We believed there were still too many stars within the potential extremely metal-poor area, so we further reduced the error limits to 0.2 for the y-axis and 0.1 for the x-axis, making sure to only use the reduced catalogue rather than the whole data set. The average errors in the axes were again given to the coding team and we plotted our newly reduced data against the coding team’s newly altered data to give the figure below.

A plot of stars (reduced using their errors) overlaying the theoretical clouds produced by the coding team.

This graph shows that there are a lot less potential extremely metal-poor stars, but the coding team needs to increase the x-axis range of the cloud so that the data can be cut more accurately.

The Coding Wizards

Picard and Data prepared themselves for a confrontation with their own creations, thousands of points generated by the very thing they hoped to destroy: The Errors.

This week we aimed to firstly produce code that would, according to a Gaussian distribution, generate a random spread of points for each theoretical data point, using the error (σ) that was determined from the actual data. This theoretical ‘cloud’ of stars would then be available to be overlaid over our actual measurements, thus allowing cuts and reduction of the data to only the most metal-poor stars that were in similar locations to the theoretical metal-poor star ‘clouds’.

Picard never did like seeing merely numbers, in a brief reprieve between combatting the ever-looming error threat, he took a trip to the holodeck where he could gaze to his heart’s content upon a beautiful star of his own creation.  

While waiting for the error in the actual data to be provided, we produced a plot of the NB392 filter and g filter over the spectrum of a F-type star.

The spectrum of an F-type star overlayed with NB392 and g filters.

As can easily be seen that the Ca II K absorption line is present within the range of the NB392 filter’s range. 

As well, we produced a plot of our theoretical data using the axes we have finally decided upon using: (NB392-u)-(g-i) against g-i. 

The theoretical data plotted as (NB392-u)-(g-i) against g-i.

Upon receiving errors for both axes, σ(NB392-u)-(g-i)=0.16149 and σ(g-i)=0.08831, we were able to produce a plot of the theoretical distribution after each star was modelled as a Gaussian for 4000 data points.

Picard gazed upon the deep red smudges and couldn’t help but think of the blood of his crew which would cover his hands should this mission fail. This thought troubled him greatly. He locked it deep within his labyrinthine mind. Thoughts like that at such a crucial time would only inch him further towards he and his loyal crew’s undoing.

The theoretical clouds of stars produced using the errors given by the Theoreticians.

Later, we were provided with more strictly determined errors, σ(NB392-u)-(g-i)=0.106753 and σ(g-i)=0.56765, which allowed us to produce the following distribution plot of our theoretical data. 

The updated theoretical clouds of stars produced using the new errors provided.

As can be seen, the respective metallicity regions are more defined with these new errors. 

We spent a short while determining how to correctly use the “Polyfit” python module, as we were informed this would be a possible method of creating this line by our mentor. 

After this, we decided to attempt different methods to make the cut line, one would be to create a Polyfit line which would fit to the mean points of all the closest points -5.0 and -4.0 metallicity stars.  

The other would be to generate a colour map for the metallicities of the stars. Using our theoretical stars, the metallicities of these stars were assigned to the z-axis, which had a Gaussian fitted to then smooth out the single points. From this, we generated a smooth map of different metallicity points. This plot can be seen below. All future stars can be mapped against this plot and their metallicity calculated from where they lie with respect to the colour map.

A colour map for metallicities generated from theoretical stars. The points are the original stars that were used to calculate the map.

We aim to create this line over the following week in preparation for the next lab session. We will also calculate the error in the metallicity from the error in colour given to us. From this, we should be able to pass the observational data, calculate the metallicities (with errors) and thus make more cuts.

Aside: LCdr. La Forge expressed discontent with Cdr. Riker’s choice of biscuits for the weekly tea break, noting that the orange-flavoured Earth snack, known as “Jaffa Cakes”, were not biscuits but, in fact, cakes. The ship’s stockpile was, nonetheless, completely cleared in a matter of minutes.

WARP – Week 2

Captain’s log, stardate: -303894.

The Theoreticians

Doubt began to cloud Commander Troi’s finely-tuned mind. As a Betazoid, she’d always been strongly aware of such feelings. She felt that, while the data reduction performed by the Theoreticians had been executed as planned, it could further be improved.

We firstly decided to move the cut in our data slightly to the left of its previous position to include more galaxies, meaning there is now less contamination of galaxies in the stars section which is the portion that is important for later use. The new cut along with its effects are shown on the contamination graph below.

The new contamination plot with a cut that further reduces the amount of galaxies in the stars portion.

Star data was given by the Coding Wizards in the form of a table and a new column was added for the temperature of each star. This was approximated using the B-V colour index in the formula below which assumes the stars are perfect black bodies. A subset was also created within TOPCAT which separated the stars into their various spectral types via their temperature.

The equation that was used to calculate the temperature of each star.

We also wanted to calculate the volume of space we are surveying per spectral type so we can later use them to find the metal-poor star density per spectral type. The volumes have to be calculated separately as stars are observed as different distances, for example M type stars are dimmer than O type stars so we can’t see M type stars at as large distances. As the data from the Coding Wizards included the stars’ distances, we could find the limiting distances (the furthest star of each spectral type) to calculate the volume.

As the pilot of the ship, La Forge deemed it his duty to know the space he would be navigating – the lives of her crew were in his hands after all.

The area we are surveying (approximately 1.9 square degrees) was used to create a ratio with the total square degrees of the Earth (approximately 41,000 square degrees). The surveyed area could then be worked out for each spectral type by multiplying this fraction by the volume of a sphere where the radius of the sphere was equal to the limiting distance of that type.

As leader of the Theoretician away team, Commander Riker felt it necessary to evaluate every possibility and eventuality – his Captain had taught him well. If any errors were to befall his loyal crew, he would always ensure that he were at the forefront of the fight against them.

Towards the end of the lab session, we started to look into the errors of the stars’ magnitudes. Using errors of the measurements of NB392, U, g and i, we can get an overall error in ‘(NB392 – U) – (g – i)’ and ‘(g – i)’ for each of our stars. The average of these will be taken over all the stars and this will then be given to the coding team to work with.

Whilst the three-person away team tackled the data head-on, Guinan immersed herself in the endless fields of knowledge housed within the Starfleet archives, tirelessly researching our elusive subjects as they continued to hide in plain sight.

She worked hard on forming the start of the report, researching topics used to create an introduction for our report that is to be produced towards the end of the 10 weeks.

The Coding Wizards

After the act of espionage uncovered in our last episode, the fear of a traitor in the midst still gripped Picard – nothing at the academy had prepared him for this – but of course he could not show weakness in this time of great strife. Via secure comms, Picard and Data were advised by Starfleet Command (Dr. Sobral) to be more self-sufficient, using their years of expertise to formulate their own refined method of magnitude calculation.

The session started promisingly with the quick refinement of the code that generated the magnitudes in different filters from the stellar spectra. By creating a spectrum for a sun-like star with a set luminosity, we could find a more accurate ratio for the calculated luminosity and the correct one. 

From the increased accuracy, a new set of colour-colour theory graphs were created.

(NB392 – U) – (g – i) against (g – i). The reason for the more compicated y-axis was to make the lines flatteras they are normalised to the colour (g – i).

At the same time, the limiting distances to spectral types of stars were calculated. The minimum apparent magnitude that can be detected by most telescopes is around 25.5. Since different spectral types have different brightness, O types being the brightest and M the dimmest, the maximum detectable distance for each type of star is different. By taking the absolute magnitude for a set of different stars and a minimum apparent magnitude of 25.5, the distances were calculated and handed to the other half of the team. The goal is to calculate the volume of the sky we are looking at for each type of star.  

Magnitudes at a distance in parsecs for different spectral types. Looks a bit like a printer error.

As Riker boldly faced the onslaught of errors with their eyes trained on the away team, a secret contingent had snuck their way past our cohort of brave explorers and right into the data!  

The next part of the lab was plotting the theory data against the observational to see what cuts could be made. Unfortunately, the observational data was exceptionally spread out and as such we couldn’t make any real analysis. One of the major reasons for this is the fact that the observational data has errors. The magnitudes detected are not exact and as such the data points should really be more like a ‘circle of uncertainty’ (which Picard noted sounded very much like himself).

This gave the Coding Wizards a new-found sense of purpose as they needed to create even more code.  

The first thing we need to do is take each data point in our theory data and generate a random spread of points according to a Gaussian distribution. (The sigma for this distribution will be found from the sigma in the observational data). The concept is to make a cloud of data points for each possible magnitude. This will then give us a theory plot with a larger spread of values to consider the errors in the observational data. Hopefully, then, we will be able to compare the theory and observational points and find cuts due to metallicities. From there we should be able to identify some metal-poor stars.  

Next time we will continue to make this distribution code, figure out an efficient way to plot it using python and make more pretty graphs.  

Using the first-hand information on the errors gathered by the away team, Data hoped to synthesise his own errors. His soulless, calculating mind knew that they could not hope to defeat the errors – his aim was instead to harness them for use in the holodeck. Captain Picard and Data readied themselves. The holodeck, filled with hordes of cloned errors, eagerly awaited them…

WARP – Week 1

Captain’s log, stardate: -303916.

Captain Picard boldly utilised the Napoleonic phrase of “divide and conquer” he had heard often during his Starfleet training by cutting the problem in two and tackling each separately. His team had two contingents: The Theoreticians and The Coding Wizards.  

Here is the progress of each of the teams:

The Theoreticians

Our theory team began by deducing conditions which can reduce the number of bodies in a catalogue. Doing this means that we can then use the reduced catalogue to look for metal-poor stars in more detail.

The first step was to find a way to filter out galaxies from data as we just want to look at stars. This was done by uploading the INT data of over 120,000 bodies into TOPCAT and creating subsets containing bodies we deem to be stars and galaxies, using their observed redshift, z, where z=0 for stars and 0.5<z<7.0 for galaxies. We then plotted these subsets on a graph where both axes were magnitude differences (e.g. J-K against B-V) and tried different combinations of colour bands (magnitudes) until we found a plot which gave the best separation of stars and galaxies. The best separation was U-J against J-K so we placed a cut at (where we deemed to be) the best division of groups. This graph is shown below.

Two sets of magnitude differences create a division of stars, in blue, and galaxies, in green, which can be best separated with a cut, ss.

It can be seen on this graph that there is contamination of stars in the galaxies portion (under/to the right of the cut) and vice versa. This contamination is shown for the whole set of data in the figure below.

The contamination of galaxies in the stars portion, and vice versa, from the sample, can be clearly shown over the whole dataset.

Next, we created various subsets to filter out galaxies and data points which had been given extreme values (bodies that are too faint to observe) which were then applied to the whole catalogue and created a graph containing only the possible metal-poor stars.

When the galaxies are filtered out using subsets, only stars are left which we can then use to potentially find metal-poor stars.

The Coding Wizards

Our coding team, consisting of Picard and Lieutenant Commander Data, were the first of our intrepid explorers to encounter a problem, materialising in the form of the code producing totally unrealistic results.

For example, we considered and processed data for a sun-like star (G type, Temperature=5000K) and received absolute magnitudes of approximately -80, which is highly physical. The expected values are about 4.8. We attempted many different ways to resolve this problem such as analysing the units of the produced result to see whether we had made an error when we created the mag output equation, however, there was no such resolution this way.

Upon consulting our mentor D. Sobral, we then checked that our code would produce the correct luminosity of a sun-like star at 10pc. Our value was an order of 18 off the correct value. This allowed us to determine that the cause of our errors was due to the assumption that we had made about the data. We had assumed it would be flux at 10pc and produce absolute magnitude.

Picard and Data believed treachery to be afoot!

Checking the luminosity our data produced for a Sun-like star (assuming the star to be at 10pc) allowed us to determine the ratio between the expected value of luminosity and the value we observed, multiplying our calculated flux by this ratio allowed us to correct for our error and receive sensible results.

Picard and Data’s fears were confirmed as the measurements of flux were found to use units of measurement unlike any found in Federation space, leading to speculations of espionage…

We then processed the data for a variety of stars of varying metallicities, ranging from -5.0 to 0.0 and started to produce theory graphs.

The first attempt at a colour-colour theory graph for spectral types and metallicities.

In conclusion, we had a little bit of a scream, but it all went pretty well for the first week. We’ve all mostly gotten to grips with the software and the code respectively, and we’re all set to keep up the good work next week.

Next week Picard and Data will continue to refine and perfect their data and produce a series of theory colour-colour graphs on the metallically and spectral types of stars. These will be utilised by La Forge, Riker and Troi to determine which of our vast catalogue of bodies could indeed be metal-poor stars.  Guinan, meanwhile, shall delve into the history of metal-poor stars, their inception and their subsequent discovery, for if we are ever to find these elusive celestial inhabitants, we must first truly understand them…