Finding patterns in data

News

September 2013

Yes, we are still here! Things are happening slowly with V4 of PATN. Yes, V4. There have been many new ideas that we have wanted to implement, but we all had a few other Blatant jobs that always seemed to take priority. We are hoping that V4 makes it out before the end of the year.

April 12, 2010

We are working on a minor update to bring all previous versions into line in regards random number generation. We picked up minor inconsistencies between PATN code running on Linux and Windows systems. A minor update (v.3.13) will be released soon.

July 15, 2009

It has been far too long I know (apologies), but a key player in the team has been in need of support through tough times. Version 3.12 is now released and is another free update to all existing PATN users (Go to File|Check for Update). The changes include-

  • 3d-plot. Improve graphics control of 3d-plot video and default to stable video codec.
  • 3d-plot. Generate the legend in a separate scrollable window.
  • 3d-plot. When opening a .ptn file that has an ordination view open from the previous session, the right mouse click menu options will now be highlighted correctly.
  • 3d-plot. User Defined Group Colours is disabled while B & W Symbols is enabled. This addresses a mismatch between the legend and the objects while B&W symbols option was enabled.
  • 3d-plot. Revised logic for the display of user-defined colours. User-defined group colours are independent of centroid colours (PATN and a-priori group).
  • 3d-plot. Enable full control of fonts.
  • 3d-plot. Text box function improved for selection, font changes and drag.
  • ANOSIM. Fixed ANOSIM looping forever in a few rare situations.
  • ANOSIM. Fixed ANOSIM export and display buttons now enabled appropriately.
  • Box & Whisker plots. Correct problem in multi-page printing.
  • Box & Whisker plots. Enable full font control.
  • Data table. When opening a saved ptn file the marginal stats for both rows and columns are made visible even if they were not visible when the file was saved.
  • Dendrogram. Label + symbol options on dendrogram now display number in group.
  • Dendrogram. Correct number in group of Column group (groups-only) display.
  • Evaluation. A-priori group option added to ANOSIM and box and whisker plots.
  • Evaluation. Fixed a-priori group handling when number of groups different to PATN groups.
  • Export. Date and time now displayed in exported Kruskal-Wallis file.
  • Export. Ordination emf format export improves display of PCC variables.
  • Export. Improve graphic images from 3d-plot (some symbols dropped on emf).
  • Histograms. Enable full font control.
  • Import. The import of DOS PATN archive files improved.
  • Import. Improve the import of PATN relational format files.
  • Installation. PATN will hopefully no longer 'disappear' for a long period of time without comment during first PATN activation. PATN takes a while to detect its environment.
  • Non-hierarchical classification. Object-centroid distances are now tabulated on group tab and exported on row group composition file. This enables easy evaluation of ‘representativeness’ in environment conservation.
  • PCC. Improved handling of missing data.
  • Tools | Options | Black & White Symbols are now better aligned.
  • Fully updated Help file, of course.

June 4, 2008

Yes, it has been a while since we have done an update, as ever. We are working on releasing version 3.12 soon. This version contains may small fixes and a range of added functionality. Stay tuned.

August 15, 2006

  • A conscientious user identified a problem with 'Data | Create new variables from variable groups'. What should have taken seconds with certain configurations was taking hours, due to a few missing lines of code :). This problem has been fixed in PATN v3.11. This is a free update for existing users (run File | Check for Update).
  • A user has requested a list of references and related resources for the web site. We are working on it.

July 28, 2006

  • A free downloadable trial version of PATN is now available. This version contains the 'Medals' dataset and is fully functioning except that the demo version it will not import a Data Table, but will import evaluation data
  • Added a demonstration 'use-case' on marine benthic community data.
  • Added a FAQ on the PATN installation process.
  • Added a FAQ on Data Management and PATN.
  • PATN Discussion Forum: There have been a number of malicious break-ins on the bulletin board from what appears to be an Indonesian source. I recovered everything so no losses. As a consequence of the break-ins, I have had to increase the level of security on new members. This means that I will now validate all new members. This doesn't affect current members.

June 2, 2006

PATN v3.1 released! The changes include-

  • PATN can now import a lower symmetric association matrix for objects or variables (File | Import | Row (or column) Lower Symmetric Association matrix). This matrix can be either be formatted as an unlabelled CSV file or as a file in the identical format as exported from PATN. A classification and ordination can be performed on this matrix by selecting Association = "None" in the Data Analysis Window. All subsequent evaluations (e.g. ANOSIM, displaying association on the ordination plot etc) will use the imported association matrix.
  • A Print Preview button has been added to the standard button bar.
  • A Print button has been added to the standard button bar.
  • A new option has been added to the Data menu that will create one new extrinsic variable for each variable group. This option is called "Create new variables from variable groups". The new variable for each variable group is the mean of the values in each of the variables within the group. For example, the new variable Group 1 will contain N rows that will be the mean of each variable in variable group 1. This option helps to reduce noise on datasets with a large number of variables. The method produces new variables which are groups of original variables.
  • The export of column (variable) group statistics has been added to the File | Export Evaluation Data menu. Export group statistics has been renamed Export Row Group statistics.
  • Two-step association for column association replaces Bray and Curtis where it appears that the variables are species and the matrix is sparse.
  • The time required for each step in an analysis is now added to the log file.
  • A non-hierarchical classification (rows only) will now produce a correct two-way table. The ordering of rows within the groups is arbitrary, but a user-defined ordering of the rows within the table can be imported (see next item).
  • The ordering of rows or columns in dendrograms can now be imported as a CSV file (File | Import | Row (or Column) Dendrogram Order. Imported ordering will be used for displaying dendrograms and two-way tables. It is possible to produce an ordering of the dendrogram which, given the classification, is impossible to display correctly. If an 'illegal' ordering is imported, PATN will still try and display a dendrogram but it may look a mess! If a legal ordering is used, the dendrogram will display correctly.
  • The row and column dendrogram ordering is appended to data table on export (if they are available).
  • PATN will now detect duplicate rows in the Data Table (Data | Delete duplicate rows) and allow for deletion of duplicates.
  • The default 'Visible Stats' tab will now display 'Num > 0' instead of 'mean' by default. The display can be changed using Tools | Options.
  • The missing data value is now set to "-9999.9999" when a random Data Table is generated in PATN.
  • The labeling of the ANOSIM output has been corrected. The values output were correct, but the labeling wasn't.
  • The layout of the dendrogram window has been improved to scale better.
  • The ordination plot legend is now generated to a scrollable region. The text and symbol size can be controlled in that window as with other PATN windows.
  • Page-Up and Page-Down keys now control symbol size on the Ordination Plot and the scatter plot. Try it. Neat!
  • Multiple monitors will not work with PATN's Ordination Plot - but PATN won't fall in a heap either.
  • Unhide of a row or column will work when any adjacent row or column is selected and unhide is activated.
  • Transposition will now reset extrinsics and hidden rows and columns.
  • Improved capture and display of BMP ad EMF images.
  • Eliminated PATN's prompt for saving the project file when no changes had been made to it.
  • A long Project Comment will now automatically wrap in the text box.
  • Status bar is updated faster when changes are made to the number of rows or columns in the Data Table.
  • Importing ordination coordinates will update the Ordination Plot.
  • View | Ordination was cross-linked to View | ANOSIM - now corrected.
  • Corrected overflow when applying Box and Whisker plots to a large number of objects.
  • Sparse data in 'relational' format can now be imported into PATN. Data with >50% zeros is often stored in the form 'species label, site label, abundance value'.
  • Imported ordination coordinates can be used in the 3d-display even if an ordination had not been previously run.
  • An error in the display of a-priori centroid labels in the 3d-plot has been corrected.
  • An error in the display of the lengths of the ordination axes has been corrected. In v3.01-3.03, the ordination axes were displayed with unit length.
  • The 'depth' (z-axis) of the 3d ordination display has been increased to improve its 3d properties.
  • Ordination 'spheres' have been rendered with a finer mesh to improve the view.
  • 3d-plot legend now uses scrollbars as required.
  • ANOSIM has a new option that can use either PATN generated groups or an a-priori grouping (defined externally to PATN).
  • Box and Whisker plots have a new option to use either PATN-generated groups or a-priori groups
  • A Zoom toggle has been added to the 3d-plot
  • Selected points in the Scatter Plot will now be highlighted in the Data Table in the same way as with the ordination plot.

December 3, 2004

  • PATN v3.03 released.
  • Added a demonstration 'use case' on the 'medals' dataset
  • Minkowski Series (Manhattan and Euclidean distance) association added to non-hierarchical classification. This series allows for weighting of variables akin to the DOS PATN ALOB routine.
  • The display of PATN and a-priori groups on the ordination plot is now annotated in the information region of the ordination plot
  • The a-priori and PATN group compositions are now displayed in the Visible stats area of he data Table (using the Groups tab).
  • Exported files now have a standard header comprising "Table Comment", "Creation Date and time" and column headers, along with other specific components such as "Association method", "Classification method" etc. Import of files using the headers and 'headerless' are both supported.
  • Long labels fixed in displaying the association matrices and fusion tables
  • Switching between window views no longer re-sizes the windows
  • Smoother scrolling of two-way tables
  • Auto-width option of columns in the data Table now considers the labels and values
  • Import and display of ordination coordinates fixed
  • Unassigned PATN or a-priori group displayed as missing value “..” and handled on import/export as “0”.
  • Kruskal-Wallis calculations are now performed in extended precision. Ah, the joys of working with millions of objects!
  • Better handling of missing data in Box and Whisker plots
  • Go to extrinsic row/column improved operation
  • Improved printing of the data table
  • The online Help has been updated to reflect changes and additional pictures have been added

October 26, 2004

  • 17 brief video clips of a range of PATN operations have been added to the Web site. These are intended as an interim until a trial version of PATN is released (replaced now by PATN's trial version).
  • I am planning to go to the Ecological Society of Australia conference in Adelaide (December 7-10, 2004). It would be good to catch up with any new, old or potential PATN users who may be attending.
  • We are working on version 3.03 a hope for a release soon.

September 25, 2004

  • I'm working a set of small video clips for most of the standard PATN operations. I am hoping that these clips will give clients a simple demonstration of some of the operations in PATN.  These clips will be added to the Web site within the next month.
  • We added some extra functionality to PATN v3.02 and are in the process of testing. Watch this space, and the Discussion Forum for details on v3.03.

September 9, 2004

  • We have been working very hard to get a range of features and fixes into PATN V3.02. While we would have liked to include more features, we wanted to get this release out to address some issues. We hope that you will like the result! Here is what has been added for PATN v3.02-
  • PATN can now handle millions of objects. Size and speed will be dependent on the size of the dataset and the system memory available for PATN.
  • Better import from the old and new DOS versions of PATN. The binary files (.prm, .clb, .rlb and .dat) are now better handled.
  • Better handling of import of ASCII (.arc) files from old and new DOS versions of PATN. Importing these files is fraught with danger due to the potential complexity of user nominated formats. We now use 'free format' for the data component (from record 5 onwards). This means that all values must be separated by at least one space or comma. See PATN v3.02 help.
  • Automatic group statistics not generated when number of groups = 1. This makes for much faster imports and random number generation when creating new test datasets.
  • Import of datasets with numeric row or column labels will display without trailing zeros.
  • One-dimensional ordinations are now available!
  • Importing 1, 2 or 3-dimensional ordination coordinates is automatically detected by PATN.
  • All EXPORTED data from PATN now includes headers and labels. All IMPORTED analysis data can be identical in format to the exported files (with header AND labels), or not have ANY headers and labels. PATN will detect and act accordingly. You can't mix and match though.
  • An exported Data Table no longer contains un-initialized ordination coordinates and PCC results.
  • File Import and Export of a-priori (user-defined) groups added and the format for export and import is the same as for standard group compositions.
  • Row labels can now be dragged to any suitable width.
  • Group numbers for all classifications now goes from 1 to k, where k = number of groups. Ungrouped objects are still labeled -1. We may change this to "0" in PATN v3.03.
  • PATN project files (*.ptn) can be dragged and dropped on PATN for import. Also double-clicking "ptn" files now works more consistently.
  • Scatter plots of any two columns (or rows) are now available through a button on the PATN toolbar (to the right of histogram button) and the View Menu. Symbol size can decreased and increased using the "," and "." keys respectively.
  • Alternate graphics options for ordination plotting has been dropped. PATN now only uses DirectX7.
  • In the ordination plot, the right mouse menu has been re-ordered into a more logical sequence.
  • In ordination Plot, the right mouse click menu: 'Display Grouping" renamed to "Display centroid colors". This makes the command more obvious. The colors of the groups are defined by the colors of the centroid of the groups.
  • In ordination plot, an option for which group type to display has been added. If "Display centroid colors" is active, the PATN groups and the a-priori group colors can be toggled on the display by using the "t" or "T" key. This is neat for detecting differences between two classifications!
  • Selecting the Data | Row | a-priori groups menu command now will automatically close any open row dendrogram window. Previously, there was a cryptic warning message in the a-priori dialog box.
  • An open ordination plot window will be automatically closed if you run a PCC evaluation.
  • All File | Import Analysis Data will add a line to the History log.
  • Check for updates from inside PATN v3.02 will  now use you Web Browser due to some problems encountered by some users behind firewall/proxy-servers. The joys of security!

July 2004

  • FAQ on PATN updates added
  • PATN v3.01 released
  • You can now import data from DOS PATN using either the x.prm, x.dat, x.rlb and x.clb (binary) files or from an 'archive' file x.arc containing parameters, data, row and columns labels in ASCII format. This was a real "DUH!!!!". Sorry guys - I hadn't run DOS PATN for ages. Mia culpa. A good thing ones of the community sent me an e-mail. Just goes to show you - if something's not right - please le me know!
  • A major revision of the Help file has been completed. There are images/screen shots for more than 70% of the help topics. New topics have been added, such as the analysis of larger datasets, random number seeds and more on the toolbars to name but a few. Extensive Browse Sequences have been added to enable the browsing of related topics or for reading the complete help text as a textbook.
  • The inbuilt logic for the default handling of larger datasets has changed. When a large dataset is detected, PATN will uncheck the running of pair-wise association (a new check box) and SSH, and select non-hierarchical clustering with an appropriate association measure (calculated as object to group centroid). These defaults can be overridden as required, but caution is required. An FAQ on large datasets has been added.
  • Links from PATN to the Web site and the PATN Discussion Forum have been added to the Help Menu.
  • An emf (enhanced metafile format) has been added to Save Image options. This format gives high resolution visuals and permits editing of graph elements.
  • Progress bars have been updated to better show the phase of analysis for larger datasets. For smaller datasets, these progress bars will operate so fast, you probably won't see them. A cancel button has been added to progress bars to opt for an "Oops - I didn't mean to do that!" for larger datasets on the options in PATN that could take a while (SSH and MCAO in particular).
  • The non-hierarchical algorithm has been rewritten to optimize speed and two stopping rules  implemented (maximum number of iterations, and a minimum number of object relocations between groups per iteration).
  • The View Menu has been re-designed to add and cluster similar Evaluation options together.
  • The term "Kruskal-Wallis" has been replaced by "box and whisker plots" in the View menu and other relevant areas of PATN and the documentation. The philosophy is that requesting box and whisker plots will generate group statistics of which the Kruskal-Wallis test is one.
  • The toolbars have been relabeled 'standard' and 'PATN'. The standard toolbar is above the PATN toolbar, and contains buttons for file-level, Data Table manipulation, and the three main sub-components of PATN (transform/standardize, analyze and evaluate. The PATN toolbar contains the key View options of PATN. In both toolbars (and he View menu), the buttons have been re-ordered to better reflect the logic of PATN. The image on the button for View PCC is now two vectors, while MACO is the character "M" and ANOSIM the character "A".
  • The Status Bar now displays the number of objects (rows) and variables (columns) within the Data Table. Very handy!
  • Annotation of the output for ANOSIM is more explicit.
  • The generation of a Data Table with uniform random variates now uses "Var" instead of "Attrib" as column labels.
  • A View PCC button has been added to the PATN toolbar and a parallel command to the View menu. This option will display the tabulated PCC results through the PCC TAB on the Data Table. Viewing the vectors can also be viewed through the Ordination Plot (right mouse button menu).
  • Box & Whisker toolbar button operates correctly when no evaluation has been run.  The generation of box and whisker plots is now consistent with PATN's analysis philosophy.
  • If extrinsic variables are defined, and "All Evaluations" is checked on the Analysis options window, then ANOSIM, box and whisker plots, PCC and MCA are run for all extrinsic variables (automatically). If no extrinsic variables are defined and All Evaluations" is checked on the Analysis options window, then ANOSIM, box and whisker, PCC and MCAO are run for all intrinsic variables (automatically).
  • The format of the exported group statistics data now allows for better manipulation in MS Excel™.
  • View History now has a Clear Button.

June 2004

  • PATN Web goes live
  • The PATN web site was launched. It has been designed to be simple, yet functional, and will evolve with input from the PATN community.
  • PATN for Windows available for download. The first release version of PATN for Windows is available from this web site. The price of US$349 is around 35% of the price that CSIRO had previously been selling DOS PATN.